Cloud / GCP / Go / Programming

Working with go modules and GCP

Go modules are a tool to include dependencies with your project. They do not replace the need for the GOROOT or GOPATH, but they help managing the packages that you use, like npm in NodeJS. I myself prefer the usage of Makefiles, but, the module files have the advantage of standardizing everything, and, when dealing with cloud functions, it is very helpful.

Go support

Currently AWS and GCP(surprise) support go. MS Azure does not. Both will let you set environment variables, and you can set IAM roles as needed.
You can not share lambda (or function) core functions across clouds, but you can create libraries and execute them. Go execution in GCP is striegh forward with native structures like so:

func GetUser(w http.ResponseWriter, r *http.Request) { }
Code language: Go (go)

Both structures are well known and used. AWS is using its own structure with lambda event details. In this example, I’ll use GCP.

Creating the mongo module for function usage

Under any folder you want (but not under GOROOT or GOPATH) open a new folder named mongo. The name is important!
Initialize the module:

go mod init talreg.com/mongo
Code language: Bash (bash)

Yes, you must have a domain in the beginning, and no, it doesn’t have to be this one. The name after the domain has to be this one. You should now have a go.mod file under the mongo directory. And in case you are wondering, we do check this file to cvs. Here is the content of this file on my drive:

module talreg.com/mongo go 1.15 require go.mongodb.org/mongo-driver v1.4.4
Code language: JavaScript (javascript)


Next create another file, main.go, and type the following:

package mongo import ( "context" "fmt" "net/http" "os" "time" "go.mongodb.org/mongo-driver/bson" "go.mongodb.org/mongo-driver/mongo" "go.mongodb.org/mongo-driver/mongo/options" ) type User struct { User string `bson:"user"` Name string `bson:"name"` Address string `bson:"address"` Codes float64 `bson:"codes"` } func PullUser(w http.ResponseWriter, r *http.Request) { var user User server := os.Getenv("SERVER") if server == "" { fmt.Fprintf(w, "can't get this server: %s", server) return } mongoURL := "mongodb://" + server + ":27017/" ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() client, err := mongo.Connect(ctx, options.Client().ApplyURI(mongoURL)) if err != nil { fmt.Fprintf(w, "error connecting to the server at %s", server) } if err = client.Ping(ctx, nil); err != nil { fmt.Fprintf(w, "fail pinging to the server at %s", server) } results, err := client.Database("test").Collection("logs").Find(ctx, bson.D{}) if err != nil { fmt.Fprintf(w, "%v", err) return } for results.Next(ctx) { results.Decode(&user) fmt.Fprintf(w, "%s,%s<br>", user.Name, user.Address) } }
Code language: Go (go)

Something to note here: the main file name doesn’t need to have the same name as the package name,

Let’s now initialize the module list:

go mod tidy
Code language: Bash (bash)

This will create a go.sum file. I don’t check in this file in. For GCP usage, we will need the actual packages, so lets pull them in to our local folder:

go mod vendor
Code language: Bash (bash)

You now have a vendor (yes, vendor, singular) folder, containing all the dependencies of your code.

Uploading your function to GCP

  1. Zip together your folder content (inside the folder, do not zip the mongo folder, but it’s content) (zip -r mongo.zip *)
  2. Go to https://console.cloud.google.com/functions and create a function
  3. The function name is not critical, but it should mean something.
  4. What important is: the region and network settings ( in case you actually want to connect to a server). If you have a VPC with a mongo instance for testing, that will be the best option. Configuring your VPC security settings is beyond the scope of this post.
  5. For the environment variables, add SERVER and the address of the server. Note that you might need to set connection parameters. Even if you don’t have a server, you can put a random ip and you should get a timeout message, but the library loading will succeed.
    Note: There are two sets of variable! Build and run! You need to set this variable in the run section.
  6. You have to set your http preferences. I check no authentication, as this is test that will be deleted. If you have function to VPC connection, you can use it.
  7. We don’t need a long timeout! Select 7 seconds.
  8. Next chose go for runtime
  9. Make sure that: You type “PullUser” in the Entry point field! It would not work with the default “HelloWorld”, and for some reason it is hard to change.
  10. Choose zip upload as the source, and upload you zip file. you can use any bucket, I have one just for lambdas.
  11. Click deploy and wait a few minutes(!)
  12. You can now test your function. Even if you’re getting the error “error connecting to the server at”, your function works and loads the library correctly. If you don’t get anything and you do have a mongo server at the SERVER address, you’ll need to put some data into the logs collection in the test database:
db.logs.insertOne({ user:"jd", Name:"John Doe" ,Address: "something 96", codes:100.1 })
Code language: JavaScript (javascript)

You can execute the query multiple times, with same or other values, you will be able to see all the records (poorly formatted, but still).

Leave a Reply

Your email address will not be published. Required fields are marked *