August 1, 2020

Go io/fs Design (Part I)

As usual, LWN has a good write up on what’s going on in the Go community. This week’s discussion in on the new io/fs package. The Go team decided to use a Reddit thread to host the conversation about this draft design. LWN points out that posters raised the following concerns:

We added status logging by wrapping http.ResponseWriter, and now HTTP/2 push doesn’t work anymore, because our wrapper hides the Push method from the handlers downstream. / It becomes infeasible to use the decorator pattern more
Doing it “generically” involves a combinatorial explosion of optional interfaces

Ultimately, Russ Cox admits, “It’s true - there’s definitely a tension here between extensions and wrappers. I haven’t seen any perfect solutions for that.”

August 1, 2020

Links for the Week

The big thing from this week is the launch of the Perseverance Rover to Mars. In the last Month, United Arab Emirates, China, and United States all launched rovers to Mars. The US rover is the Perseverance. It carries a novel flying drone. It will also attempt to safely store soil samples that can eventually be brought back to US.

It makes my heart twinge even writing about it.

Atlas Obscura has an article on the origins of the automatic rice cooker. I read it over coffee this morning and thought about how happy good rice makes me feel. The humber Datung rice cooker was so central to my childhood.

July 26, 2020

Localstack S3 and Go

I spent too much time Saturday getting the Go S3 SDK to work with LocalStack.. It turns out that if you are using LocalStack, you need to explicitly configure the following properties:

	sess, err := session.NewSession(aws.NewConfig().
		WithEndpoint(endpoint).
		WithRegion(region).
		WithS3ForcePathStyle(true))

The requirements for Endpoint and Region are obvious. If S3ForcePathStyle is not specified, then LocalStack will fail.

	data, err := svc.GetObject(&s3.GetObjectInput{
		Bucket: &bucket,
		Key:    &cfgKey,
	})

What is path-style? In May 2019, Amazon deprecated path-based access model for S3 objects. This means one should no longer use URLs of the form:

July 12, 2020

Running Private Docker Registry

I still find it hard to believe how easy it is to run your own infrastructure in the cloud.

Running a Docker registry is as simple as adding a few lines of code to your Terraform configuration.

	resource "aws_ecr_repository" "foo" {
	  name                 = "bar"
	  image_tag_mutability = "MUTABLE"

	  image_scanning_configuration {
		scan_on_push = true
	  }
	}

When deployed, this create an registry where you can manage multiple Docker repositories. You can upload a Docker image to be used in your private ECS cluster:

May 26, 2020

Gomock Tutorial

I’ve been using mock/Gomock to write tests in my personal project. When you’re building something in a new language, it is hard to prioritize learning every tool in your toolchain. For me, I’ve been writing custom and suboptimal code for Gomock because of a nifty but undocumented API call .Do.

In many cases, I wan to match subsets of a complex object while ignoring irrelevant parts. e.g., verify a function is invoked with a list of User objects, but only verifying the email addresses. To do that in a generic way, I wrote a custom Matcher API that uses text/template to describe what parts of the object to match. Thus, my mock-and-verify code looks like:

May 19, 2020

Unit tests and system clock

It took me way to long to learn this. Your code (and their unit tests) should inject the system clock as a dependency.

An example, let’s say you have a service that writes a record to the database with the system clock.

public void save(String userName) {
	long currentTimeMs = System.currentTimeMillis();
	User user = User.builder()
	    .name(userName)
		.updateTimeMs(currentTimeMs);
	database.save(user);
}

How would you test this? You can inject a mock database instance and use it to verify that it got a User object. Great! You can verify the username is as expected. How do you verify that tricky business rule that updateTimeMS is the “current time”?

May 17, 2020

Go Project Organization

Here’s a rough layout of how I organize my Go project. Some parts are situational and some parts are essential. I’ll go over both in this blog.

A rough layout:

+ basedir
   +-- go.mod (module jcheng.org)
   +-- hello (empty)
         +-- log/
         +-- utils/
         +-- config/
         +-- models/
         +-- repositories/
         +-- services/
         +-- cmd/
              +-- hello_app/
                     +--/cmd/
                          +-- speak/
                          +-- email/
                          +-- sms/

The basedir

Situational.

February 27, 2020

Dependencies

Some past self version of me is saying, every class and function should be explicit about their dependencies, so that they are easily testable. John0 would say, “If you have a service that talks to a database, the database client should be an explicit dependency specified in the constructor. This makes the code easily testable.”

There is another version of myself from 10 minutes ago arguing it’s foolish to be explicit about everything. He’d point to this piece of code he’s just looked at:

February 19, 2020

Mocking in Go Part 2

In a previous post, I talked about Gomock. I want to spend a bit more time on it.

As I’ve mentioned, setting up Gomock requires

Download mockgen

go get github.com/golang/mock/mockgen@latest

Edit your go.mod

module jcheng.org

go 1.13

require (
	github.com/golang/mock v1.4.0
)

Add the go:generate incantation to your code

//go:generate mockgen -source $GOFILE -destination mock_$GOFILE -package $GOPACKAGE
package pastself

import (
	"time"
)
...

Use mocks in your test case. In this example, I used a callback function to match on a nested property.

type fnmatcher struct {
	d    func(x interface{}) bool
	desc string
}

func (self *fnmatcher) Matches(x interface{}) bool {
	return self.d(x)
}

func (self *fnmatcher) String() string {
	return self.desc
}

func On(desc string, d func(x interface{}) bool) mock.Matcher {
	return &fnmatcher{d: d, desc: desc}
}

// TestSendOverdueMessages_ok is an example of using matching using a callback function
func TestSendOverdueMessages_ok(t *testing.T) {
	ctrl := mock.NewController(t)
	defer ctrl.Finish()

	mockUserRepo := NewMockUserRepository(ctrl)
	mockMessageRepo := NewMockMessageRepository(ctrl)
	mockMessageSender := NewMockMessageSender(ctrl)
	log, _ := NewBufferLog()
	r := NewPastSelfService(
		mockMessageRepo,
		mockMessageSender,
		mockUserRepo,
		log,
	)

	senderUserDDB_1 := &UserDDB{UserID: "sender1"}
	senderUserDDB_2 := &UserDDB{UserID: "sender2"}
	recipientUserDDB_2 := &UserDDB{
		UserID: "recipient2",
		Profile: UserProfileDDB{
			PreferredEmail: "recipient2@example.com",
		},
	}
	resultSet := &ResultSet{
		Items: []Message{
			{
				ID:         1,
				SenderID:   "sender1",
				Body:       "body1",
				Recipients: []string{"email://recipient1@example.com", "recipient2"},
				Headers: []Header{
					{Name: "x-header-key-1", Value: "value-1"},
					{Name: "x-header-key-2", Value: "value-2"},
				},
			},
			{
				ID:       2,
				SenderID: "sender2",
				Body:     "body2",
			},
		},
	}
	mockUserRepo.EXPECT().GetUser("sender1").Return(senderUserDDB_1, nil)
	mockUserRepo.EXPECT().GetUser("sender2").Return(senderUserDDB_2, nil)
	mockUserRepo.EXPECT().GetUser("recipient2").Return(recipientUserDDB_2, nil)
	mockMessageRepo.EXPECT().FindOverdue(mock.Any(), 0, "").Return(resultSet, nil)

	matchFrom := On("user.UserID==sender1", func(x interface{}) bool {
		if y, ok := x.(*UserDDB); ok {
			return y.UserID == "sender1"
		}
		return false
	})
	matchTo := On("[]Recipient{recipient1@example.com,recipient2@example.com}", func(x interface{}) bool {
		if y, ok := x.([]Recipient); ok {
			return len(y) == 2 &&
				y[0].Email == "recipient1@example.com" &&
				y[1].Email == "recipient2@example.com"
		}
		return false
	})
	matchOutMessage := On("OutboundMessage.Body==Body1", func(x interface{}) bool {
		if y, ok := x.(OutboundMessage); ok {
			return y.Body == "body1"
		}
		return false
	})
	mockMessageSender.EXPECT().Send(matchFrom, matchTo, matchOutMessage).Return(nil).MaxTimes(1)

	r.SendOverdueMessages()
}

One thing nice about GoMock is that it generates statically typed method names for the EXPECT() statements, so the compiler can check that you’re using the correct method names. Neat.

February 19, 2020

Few Annoying Things about Go

As promised, a few thoughts about Go that is annoying.

No Generics

Code generation is tightly coupled with the tool chain. When I need to use code generation, i.e., enums and mocks, the use case can be solved with generics instead.

No lambda syntax

Scala has a lambda syntax to make it easy to work with anonymous function objects

val numbers = Seq(1, 5, 2, 100)
val doubled = numbers.map(
    n => n * 2
  )

Java has a similar syntax