spark

Spark

mise

To install all of our protobuf, rust, and go toolchains install mise, then run:

mise trust
mise install

Recommended: Add mise shell integration so that the mise environment will automatically activate when you are this repo, giving you access to all executables and environment variables. Otherwise you will need will need to either manually mise activate [SHELL] or run all commands with the mise exec prefix.

pkg-config

We use pkg-config in the build process. One way to install it is through brew:

brew install pkgconf

lefthook (optional)

Lefthook gives us an easy way to declare pre-commit hooks (as well as other git hooks) so you're not pushing up PRs that immediately fail in CI.

You can either install it through mise (above, recommended) or brew:

brew install lefthook

Once it's installed, run lefthook install to install the git hooks, which will automatically run when you did git commit. You can also run the hooks manually with lefthook run pre-commit.

Generate proto files

After modifying the proto files, you can generate the Go files with the following command:

make

Bitcoind

Our SO implementation uses ZMQ to listen for block updates from bitcoind. Install it with:

brew install zeromq

Note: whatever bitcoind you are running will also need to have been compiled with ZMQ. The default installation via brew has ZMQ, but binaries downloaded from the bitcoin core website do not.

brew install bitcoin

DB Migrations

We use atlas to manage our database migrations. Install via mise install.

To make a migration, follow these steps:

• Make your change to the schema, run mise gen-ent
• Generate migration files by running ./scripts/gen-migration.sh <name>:
• With run-everything.sh, the migration will be automatically applied to each operator's database. But if you want to apply a migration manually, you can run (e.g. DB name is sparkoperator_0):

atlas migrate apply --dir "file://so/ent/migrate/migrations" --url "postgresql://127.0.0.1:5432/sparkoperator_0?sslmode=disable"

• Commit the migration files, and submit a PR.

If you are adding atlas migrations for the first time to an existing DB, you will need to run the migration command with the --baseline flag.

atlas migrate apply --dir "file://so/ent/migrate/migrations" --url "postgresql://127.0.0.1:5432/sparkoperator_0?sslmode=disable" --baseline 20250228224813

VSCode

If spark_frost.udl file has issue with VSCode, you can add the following to your settings.json file:

"files.associations": {
    "spark_frost.udl": "plaintext"
}

Linting

Golang linting uses golangci-lint, installed with mise install.

To run the linters, use either of

mise lint

golangci-lint run

Logging

We use Zap for logging in the SO. Here are some best practices:

• Zap includes APIs both for structured logging (with key-value pairs) and unstructured logging (with fmt-style formatting), called "Sugared" and "Unsugared" respectively. Since we index structured attribute keys in our logging backend, they should be reserved for common attributes that we want searchable. For example:
  • error: we use this for logging all errors (through zap.Error).
  • identity_public_key: we use this for logging the identity of the public key that is making a particular request. It should not be used for logging one-off attributes that don't have a clear purpose or common meaning across the codebase, i.e. count, value, etc.
• Default to logger.Sugar().Infof(...) for dynamic runtime values.
• Structured fields are an allowlist, not a general pattern. If a value is not an approved searchable key, log it in the message string, not as a zap field. If you're unsure, err towards unstructured logging.
• Converting between zap.Logger and zap.SugaredLogger is easy and cheap. Use Logger.Sugar() to go from zap.Logger to zap.SugaredLogger, and SugaredLogger.Desugar() to go the other way.
• Most places in the codebase have a zap.Logger injected into the Context with common fields already present. This includes all requests, tasks, and most other major components (i.e. chainwatcher). Use logging.GetLoggerFromContext to get the logger instance from the context.
• Structured and unstructured logging can be mixed for a given log message. For example, the following example is perfectly acceptable:

	logger := logging.GetLoggerFromContext(ctx)
    ...
    logger.With(zap.Error(err)).Sugar().Infof("Failed to broadcast node tx for node %s", node.ID)