A Simple Reusable Scheduler in Go

Tagline: You (still) don’t need a queueing service.

I recently needed a module that would schedule work for a future time on behalf of users, and would execute it at said time. After some thought, I came to an easy design that leveraged the Go standard library.

Some constraints I wanted satisfied:

1. The scheduling should be driven from the database (a relational DB in my case.)
2. Users should be able to update their desired scheduled times.
3. The schedule updates should be picked up regularly, but not necessarily to-the-minute. Say, every 15 minutes.
4. The starting time of the scheduled work should be honored to the minute.

These last two items are not in conflict. Think of an airport updating the schedule of departures once every hour on the hour, though planes may be slated to leave at any time.

I also didn’t want any busy-waiting loops in my design.

I was confident that leveraging Go’s goroutines would make it unnecessary to rely on a queueing service plus yet another binary / microservice to deploy.

Representing future work

Scheduled work is represented using Go’s timers.
Specifically func AfterFunc(d Duration, f func()) *Timer does just what we want: after duration d, a goroutine will execute function f. Note that the signature func () is as reusable as could be.

Since we want to be able to change the scheduled time of a task, we need to represent the triggering time too.

Here is the basic type I defined:

// FutureAction represents some action to be taken
// at a specified future time
type FutureAction struct {
    TriggerTime time.Time
    Action      *time.Timer
}

With this structure, we can schedule, execute, and delay work.

Scheduling work

As mentioned at the beginning, the schedule should be driven from the Database, so we created a goroutine sitting in a loop, waiting for the tick of a time.Ticker to query the database for new tasks or updated execution times.

The fact that a task has been scheduled should not be written into the Database, though, but be kept in memory as part of our scheduler. This makes it much easier to reason about crash recovery or process restarts.

So, our scheduler should do, roughly (in Go-inspired pseudocode):

for {
    // on new tick from the master Ticker
    WorkToSchedule ← read work to schedule from the DB
    for work in WorkToSchedule {
        if work is overdue {
            Do work
        } else if work already scheduled && new trigger time {
            Reschedule work
        } else if work not scheduled {
            Schedule work
        }
    }
}

The nature of the actual work is not important for the structure of the loop, which means that if we represent it in a reusable manner, we have ourselves a re-targetable scheduler.

Go’s interfaces are just the thing to use:

// Work represents a task that may be
// done now, or scheduled for later
type Work interface {
    DoIt()
    ScheduleIt()
    ResetTimer(triggerTime time.Time)
    GetTriggerTime() time.Time
    IsScheduled() (bool, time.Time)
}

The implementation specific parts

What are the implementation-specific bits of this design?

1. The actual work to be performed
2. How we find/store scheduled work

As we mentioned before, the fact that some particular task has already been scheduled by our scheduler should not be written into the database. Rather, it is information that the scheduler should keep.

Imagine that the work to be done is a wake-up call in a hotel. We should be able to check whether Room 121-B’s wake-up call has been scheduled. It makes sense for the scheduler to keep a map[string]FutureAction to represent scheduled work.

We could define:

type Room struct {
    PhoneExtension string
    Number         string
}

// WakeUpCall is the Work of giving a wake-up call to a room
//
// roomRepo is the Room database interface
type WakeUpCall struct {
    room          Room
    phoneSchedule map[string]FutureAction
    phoneService  services.Phone
    roomRepo      room.Repository
}

For instance, to implement the IsScheduled method of Work:

func (wuc WakeUpCall) IsScheduled() (bool, time.Time) {
    action, found := wuc.phoneSchedule[wuc.room.Number]
    if !found {
        return false, time.Time{}
    }
    return true, action.TriggerTime
}

Performing the Work is easy. We should ensure that once it has been done, it is taken out of the schedule.

func (wuc WakeUpCall) DoIt() {
    done := wuc.phoneService.WakeUpCall(wuc.room.PhoneExtension)
    if done {
        delete(wuc.phoneSchedule, wuc.room.Number)
        wuc.roomRepo.ClearWakeUpCall(wuc.room.Number)
    }
}

Conclusion

This design is simple and reusable, as advertised. It can be easily deployed as an extra goroutine in your existing service(s).

It is not trying to solve the issue of scalability; it is not trying to schedule work among EC2 instances in a server farm, for example. But it could be a starting point for a more scalable design.

Intentionally, the design does not use queues pub-sub or any such scheme. Those are used too often already. It is always a good idea to read How do you cut a monolith in half?

Using a message broker is a tradeoff. Use them freely knowing they work well on the edges of your system as buffers.