Skip to content

sandboxset: thread-safe sandbox pool (issue #217)#425

Draft
AmiBuch wants to merge 11 commits intoopen-lambda:mainfrom
AmiBuch:main
Draft

sandboxset: thread-safe sandbox pool (issue #217)#425
AmiBuch wants to merge 11 commits intoopen-lambda:mainfrom
AmiBuch:main

Conversation

@AmiBuch
Copy link

@AmiBuch AmiBuch commented Mar 3, 2026

sandboxset: thread-safe sandbox pool (issue #217)

API

GetOrCreateUnpaused returns a *SandboxRef — a handle that wraps the
sandbox with a back-pointer to its parent set. The caller uses
ref.Sandbox() to access the container. When done, set ref.Broken = true
if the sandbox is unhealthy, then always call ref.Put() — it recycles
the sandbox if healthy or destroys it if broken.

set, _ := sandboxset.New(&sandboxset.Config{
    Pool:        myPool,
    CodeDir:     "?url=https%3A%2F%2Fgithub.com%2Fpath%2Fto%2Flambda",
    ScratchDirs: myScratchDirs,
})

ref, _ := set.GetOrCreateUnpaused()   // create or reuse a sandbox
sb := ref.Sandbox()                    // access the underlying container
// ... handle request using sb.Client() ...

if broken {
    ref.Broken = true                  // mark unhealthy before returning
}
ref.Put()                              // recycles if healthy; destroys if Broken

set.Close()                            // tear down everything

For explicit teardown with a reason, ref.Destroy("reason") is also available.

SandboxRef

type SandboxRef struct {
	set    *sandboxSetImpl
	Broken bool
	sb    sandbox.Sandbox
	inUse bool
}

func (r *SandboxRef) Sandbox() sandbox.Sandbox    // access the container
func (r *SandboxRef) Put() error                   // return to pool (or destroy if Broken)
func (r *SandboxRef) Destroy(reason string) error  // explicitly destroy and remove

SandboxSet interface

type SandboxSet interface {
    GetOrCreateUnpaused() (*SandboxRef, error)
    Close() error
}

Config

type Config struct {
    Pool        sandbox.SandboxPool  // creates/destroys sandboxes
    Parent      SandboxSet           // optional parent set to fork from
    IsLeaf      bool                 // marks sandboxes as non-forkable
    CodeDir     string               // Lambda handler code directory
    Meta        *sandbox.SandboxMeta // runtime config (memory, packages, etc.)
    ScratchDirs *common.DirMaker     // creates a writable dir per new sandbox
}

Flow

     [created]
         |
         v
     [paused]  <---+
         |         |
         v         |
     [in-use]  ----+  (Put, Broken=false)
         |
         v
   [destroyed]     (Put with Broken=true / Destroy / Close / error)

File structure

go/worker/sandboxset/
    api.go           — SandboxSet interface, Config, New()
    sandboxset.go    — SandboxRef, sandboxSetImpl, all methods
    tests/
        sandboxset_test.go              — Unit tests (MockSandboxPool)
        sandboxset_integration_test.go  — Integration tests (real DockerPool)

Dependencies

sandboxset is a thin layer on top of sandbox — no new abstractions:

sandboxset
    │
    ├── sandbox.Sandbox      (3 of 11 methods used: Pause, Unpause, Destroy)
    ├── sandbox.SandboxPool  (1 method used: Create)
    ├── sandbox.SandboxMeta  (passed through to Create, never read)
    └── common.DirMaker      (1 method used: Make)

Testing

  • Unit tests (3 tests): Use MockSandboxPool from sandbox/mock.go.
    Fast, no Docker needed.

    cd go && go test ./worker/sandboxset/tests/ -v -race -count=1
    
  • Integration tests (4 tests): Use real DockerPool with ol-min
    image. Gated with //go:build integration. Verify real containers are
    created, paused, unpaused, and destroyed.

    cd go && go test ./worker/sandboxset/tests/ -v -tags=integration -count=1
    

Next steps

  • Step 2: Replace LambdaInstance goroutines with a SandboxSet
  • Step 3: Use SandboxSet as the node in the zygote tree

Copy link
Member

@tylerharter tylerharter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I forgot to click submit on the feedback, sorry!

Copy link
Member

@tylerharter tylerharter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work! Getting closer.

Copy link
Member

@tylerharter tylerharter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

set *sandboxSetImpl
Broken bool // public: caller sets true if request failed; Put will destroy instead of recycle
inUse bool // true when checked out; false when idle in pool
destroyed atomic.Bool // set atomically after Destroy(); guards against concurrent Close + Put
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no atomic, then mutex of the containing sandbox set should be used to protect this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of destroyed, perhaps sb can just be nil?

// sandboxSetImpl is the private concrete type returned by New.
// All mutable state is guarded by mu.
type sandboxSetImpl struct {
mu sync.Mutex
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pattern I like:

things not protected by the lock
mu synt.Mutex // protects below members
things protected by the lock

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants