Skip to content

Apache Pulsar

Since testcontainers-go v0.19.0

Introduction

The Testcontainers module for Apache Pulsar.

Testcontainers can be used to automatically create Apache Pulsar containers without external services.

It's based on the official Apache Pulsar docker image, so it is recommended to read the official guide.

Adding this module to your project dependencies

Please run the following command to add the Apache Pulsar module to your Go dependencies:

go get github.com/testcontainers/testcontainers-go/modules/pulsar

Usage example

Create a Pulsar container to use it in your tests:

ctx := context.Background()

pulsarContainer, err := pulsar.RunContainer(ctx,
    testcontainers.WithImage("docker.io/apachepulsar/pulsar:2.10.2"),
)
if err != nil {
    panic(err)
}

// Clean up the container
defer func() {
    if err := pulsarContainer.Terminate(ctx); err != nil {
        panic(err)
    }
}()

Module Reference

The Pulsar module exposes one entrypoint function to create the containerr, and this function receives two parameters:

func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*Container, error)
  • context.Context, the Go context.
  • testcontainers.ContainerCustomizer, a variadic argument for passing options.

Container Options

When starting the Pulsar container, you can pass options in a variadic way to configure it.

Image

If you need to set a different Pulsar Docker image, you can use testcontainers.WithImage with a valid Docker image for Pulsar. E.g. testcontainers.WithImage("docker.io/apachepulsar/pulsar:2.10.2").

Image Substitutions

In more locked down / secured environments, it can be problematic to pull images from Docker Hub and run them without additional precautions.

An image name substitutor converts a Docker image name, as may be specified in code, to an alternative name. This is intended to provide a way to override image names, for example to enforce pulling of images from a private registry.

Testcontainers for Go exposes an interface to perform this operations: ImageSubstitutor, and a No-operation implementation to be used as reference for custom implementations:

// ImageSubstitutor represents a way to substitute container image names
type ImageSubstitutor interface {
    // Description returns the name of the type and a short description of how it modifies the image.
    // Useful to be printed in logs
    Description() string
    Substitute(image string) (string, error)
}

type NoopImageSubstitutor struct{}

// Description returns a description of what is expected from this Substitutor,
// which is used in logs.
func (s NoopImageSubstitutor) Description() string {
    return "NoopImageSubstitutor (noop)"
}

// Substitute returns the original image, without any change
func (s NoopImageSubstitutor) Substitute(image string) (string, error) {
    return image, nil
}

Using the WithImageSubstitutors options, you could define your own substitutions to the container images. E.g. adding a prefix to the images so that they can be pulled from a Docker registry other than Docker Hub. This is the usual mechanism for using Docker image proxies, caches, etc.

Wait Strategies

If you need to set a different wait strategy for the container, you can use testcontainers.WithWaitStrategy with a valid wait strategy.

Info

The default deadline for the wait strategy is 60 seconds.

At the same time, it's possible to set a wait strategy and a custom deadline with testcontainers.WithWaitStrategyAndDeadline.

Startup Commands

Testcontainers exposes the WithStartupCommand(e ...Executable) option to run arbitrary commands in the container right after it's started.

Info

To better understand how this feature works, please read the Create containers: Lifecycle Hooks documentation.

It also exports an Executable interface, defining one single method: AsCommand(), which returns a slice of strings to represent the command and positional arguments to be executed in the container.

You could use this feature to run a custom script, or to run a command that is not supported by the module right after the container is started.

Docker type modifiers

If you need an advanced configuration for the container, you can leverage the following Docker type modifiers:

  • testcontainers.WithConfigModifier
  • testcontainers.WithHostConfigModifier
  • testcontainers.WithEndpointSettingsModifier

Please read the Create containers: Advanced Settings documentation for more information.

testcontainers.WithConfigModifier(func(config *container.Config) {
    config.Env = append(config.Env, "PULSAR_MEM= -Xms512m -Xmx512m -XX:MaxDirectMemorySize=512m")
}),
testcontainers.WithHostConfigModifier(func(hostConfig *container.HostConfig) {
    hostConfig.Resources = container.Resources{
        Memory: 1024 * 1024 * 1024,
    }
}),
testcontainers.WithEndpointSettingsModifier(func(settings map[string]*network.EndpointSettings) {
    settings[nwName] = &network.EndpointSettings{
        Aliases: []string{"pulsar"},
    }
}),

Here, the nwName relates to the name of a previously created Docker network. Please see the How to create a network documentation for more information.

Pulsar Configuration

If you need to set Pulsar configuration variables you can use the WithPulsarEnv to set Pulsar environment variables: the PULSAR_PREFIX_ prefix will be automatically added for you.

For example, if you want to enable brokerDeduplicationEnabled:

testcontainerspulsar.WithPulsarEnv("brokerDeduplicationEnabled", "true"),

It will result in the PULSAR_PREFIX_brokerDeduplicationEnabled=true environment variable being set in the container request.

Pulsar IO

If you need to test Pulsar IO framework you can enable the Pulsar Functions Worker with the WithFunctionsWorker option:

testcontainerspulsar.WithFunctionsWorker(),

Pulsar Transactions

If you need to test Pulsar Transactions you can enable the transactions feature:

testcontainerspulsar.WithTransactions(),

Log consumers

If you need to collect the logs from the Pulsar container, you can add your own LogConsumer with the WithLogConsumers function, which accepts a variadic argument of LogConsumers.

if len(c.LogConsumers) > 0 {
    c.WithLogConsumers(ctx, tt.logConsumers...)
    defer func() {
        // not handling the error because it will never return an error: it's satisfying the current API
        _ = c.StopLogProducer()
    }()
}

An example of a LogConsumer could be the following:

// logConsumer is a testcontainers.LogConsumer that prints the log to stdout
type testLogConsumer struct{}

// Accept prints the log to stdout
func (lc *testLogConsumer) Accept(l testcontainers.Log) {
    fmt.Print(string(l.Content))
}

Warning

You will need to explicitly stop the producer in your tests.

If you want to know more about LogConsumers, please check the Following Container Logs documentation.

Container methods

Once you have a Pulsar container, then you can retrieve the broker and the admin url:

Admin URL

serviceURL, err := c.HTTPServiceURL(ctx)

Broker URL

brokerURL, err := c.BrokerURL(ctx)