Extensibility
Is tilegroxy's out of the box capabilities not sufficing for your use-case? Luckily tilegroxy is designed to be highly extensible so you can add whatever functionality yourself!
| If possible, please consider contributing back whatever functionality you add if it has generic usefulness. |
Overview
There are three ways to extend tilegroxy. The first is to use the various "custom" options to provide interpreted Go code to implement a provider, authentication scheme, or analytics destination. The second is to use tilegroxy as a library and create your own executable with whatever tweaks you need. The last is to fork the repository and change the code directly.
| Option | Capabilities | Limitations | Compatibility |
|---|---|---|---|
Providers, authentication, and analytics written in a single Go file per entity. No build step, the code is interpreted from your configuration |
Only the standard library and the |
Covered by the compatibility promise. Function signatures only change in a major release |
|
Every entity type, including caches, datastores, secret sources, and health checks. Any third party library can be used |
You build and release your own executable. Only |
Best effort. Changes to |
|
Forking |
Anything, including the core request flow, built-in entities, and commands |
No support. You maintain the fork yourself and merge upstream changes on your own |
None. Everything under |
"Custom"
You might have noticed "custom" listed a few times in the Configuration documentation. These options allow you to provide your own custom code that is interpreted on the fly to fulfill the specific needs you have. These custom options must be written in Go and are interpreted using Yaegi. Yaegi offers a full featured implementation of the Go specification without the need to precompile.
Your custom code must live within a single file for each provider/auth/analytics.
Custom code can use the entire standard library including potentially dangerous function calls such as exec and unsafe. Be as cautious using custom providers from third parties as you would be executing any other third party software.
|
There is a performance cost of using a custom vs a built-in offering. The exact cost depends on the complexity of your code, however it is typically below 10 milliseconds while tile generation as a whole is usually more than an order of magnitude slower. With authentication especially tilegroxy utilizes caching to mitigate this impact. However it is something you should keep in mind when deciding whether to implement a Custom provider/auth. Due to these being written in Go, it is easy to convert your custom code to a built-in equivalent if you find this overhead becomes a bottleneck.
Custom caches are not supported. A cache almost always needs an external library to talk to its backing store, which Yaegi cannot load.
Custom Providers
For cases where the built-in providers don’t suffice, you can write your own custom providers.
Example custom providers can be found within examples/providers/custom_proxy.go.
Custom providers must be within the custom package and must import the tilegroxy/tilegroxy package for mandatory datatypes.
There are two mandatory functions:
func preAuth(tilegroxy.Context, tilegroxy.ProviderContext, map[string]interface{}, tilegroxy.ClientConfig, tilegroxy.ErrorMessages) (tilegroxy.ProviderContext, error)
func generateTile(tilegroxy.Context, tilegroxy.ProviderContext, tilegroxy.TileRequest, map[string]interface{}, tilegroxy.ClientConfig,tilegroxy.ErrorMessages) (*tilegroxy.Image, error)And one optional function:
func close(tilegroxy.Context) errorThe preAuth function is responsible for authenticating outgoing requests and returning a token or whatever else is needed. It is called when needed by the application when either expiration is reached or an AuthError is returned by generateTile. A given instance of tilegroxy will only call this method once at a time and then shares the result among threads. However, ProviderContext is not shared between instances of tilegroxy.
The generateTile function is the main function which returns an image for a given tile request. You should never trigger a call to preAuth yourself from generateTile (instead return an AuthError) to prevent excessive calls to the upstream provider from multiple tiles.
The close function is for releasing any resources your provider holds, such as a database handle or a background goroutine. It will be called either when tilegroxy shuts down or during a hot reload.
The following types are available for custom providers:
| Type | Description |
|---|---|
A context.Context with special values applied. Contains contextual information specific to the incoming request. Can retrieve headers via the Value method and authz information if configured properly. Do note there won’t be a request when seed and test commands are run, this context will be a "Background Context" at those times |
|
A struct for on the fly, provider-specific information. It is primarily used to facilitate authentication. Includes an Expiration field to inform the application when to re-auth via the preAuth method (this should occur before auth actually expires). Also includes an auth token field, a auth Bypass field (for un-authed usecases), and a map |
|
The parameters from the user indicating the layer being requested as well as the specific tile coordinate |
|
A struct from the configuration which indicates settings such as static headers and timeouts. See Client for details |
|
A struct from the configuration which indicates common error messages. See Error Messages for details |
|
A struct containing the resulting imagery in a byte array called Content. You can optionally also include a field called ContentType with the mime-type of the resulting imagery. Example for how to return data: |
|
An Error type to indicate an upstream provider returned an auth error that should trigger a new call to preAuth |
|
A utility method that performs an HTTP GET request to a given URL. Use this when possible to ensure all standard Client configurations are honored |
Custom Authentication
A custom authentication works much the same way as a custom provider. The code you need to supply only has access to the standard library plus the tilegroxy/tilegroxy package. The package must be "custom" and you must include the following function:
func validate(string) tilegroxy.ValidationResultYou can optionally also define a close function:
func close(context.Context) errorThe validate method will be supplied with a single token and returns a ValidationResult with the following fields:
| Field | Type | Description |
|---|---|---|
|
|
Whether the token is valid and should allow the request to proceed |
|
|
When the authentication status of the token expires and the validate method should be called again. |
|
|
An identifier for the user being authenticated. By default this is only used for logging |
|
|
An identifier for the tenant/realm/organization/site/group the user belongs to. Omit if not applicable |
|
|
The specific layer IDs to allow access to with this specific token. Leave empty to allow access to all of them |
The method by which tokens are extracted is configurable with the token parameter. The following modes are available; if multiple are specified they’re tried in the order here:
| Order | Key | Value |
|---|---|---|
1 |
|
Header Name (in Header-Case) |
2 |
|
Cookie Name |
3 |
|
Query Parameter Key |
4 |
|
None (set as empty string) |
Close should release any resources your authentication scheme holds.
Custom Analytics
You can provide a custom analytics for recording usage events to wherever. The package must be "custom" and you must include the following function:
func record(tilegroxy.Context, []tilegroxy.AnalyticsEvent, map[string]interface{}, tilegroxy.ErrorMessages) errorYou can optionally also define a close function:
func close(tilegroxy.Context) errorAn AnalyticsEvent contains Time, LayerID, LayerName, Z, X, Y, UserID and a Fields map holding the attributes selected via fields and extraFields.
The record method is called with a batch of events, not once per request. Returning an error causes the batch to be logged as failed and discarded.
When present, close is called when tilegroxy shuts down or when analytics is torn down as part of a hot reload. It runs after the final batch has been flushed.
Using tilegroxy as a library
Tilegroxy exposes the critical classes needed to create your own executable using tilegroxy that includes your own providers, caches, authentication, analytics, datastores, secret sources, or health checks. Extending tilegroxy in this way is more complex and requires you to build your own executable but allows you to bring in third party libraries as needed and allows you to have fully custom caches.
Your executable registers your entities and then hands control to the standard tilegroxy CLI, so every command and flag keeps working:
package main
import (
"github.com/Michad/tilegroxy/cmd"
_ "example.com/yourmodule/sample" // Registers your entities from its init() functions
)
func main() {
cmd.Execute()
}To run tilegroxy commands from your own code instead of the command line, call cmd.ExecuteArgs with the arguments you would otherwise type. It can be called more than once to run several commands in sequence, such as test followed by serve. A command that fails exits the process, the same as it would from the command line.
Tilegroxy uses a registration system to find and construct its main entities. As long as you supply a struct that implements the XXXRegistration interface you can call the RegisterXXX method on startup to allow the tilegroxy internal to locate the struct. For example, here is a minimal Provider implemented in this way:
type SampleConfig struct {
// Insert configuration for your provider here
}
type Sample struct {
SampleConfig
// Add any resources your provider needs to retain through its lifecycle here. For example an SDK Client. This is shared over all requests so should generally be immutable after initialization
}
func init() {
// This registers the provider with tilegroxy so it can initialize the provider for every layer that uses it
layer.RegisterProvider(SampleRegistration{})
}
//This can generally stay empty
type SampleRegistration struct {
}
// Whatever is returned by this will be passed into the Initialize method below. If you want defaults for your configuration, set them here.
func (s SampleRegistration) InitializeConfig() any {
return SampleConfig{}
}
// This should always return the same string. Any provider configuration with name set to this value will trigger this "Sample" provider to be used
func (s SampleRegistration) Name() string {
return "sample"
}
// Declares what kind of tile this provider produces. Return config.DataTypeRaster, config.DataTypeMVT, or config.DataTypeMLT if it's always one of them, or config.DataTypeUnknown if it depends. A provider nesting others can return config.DataTypeUnknown and let its children report theirs
func (s SampleRegistration) DataType(cfgAny any) config.DataType {
return config.DataTypeUnknown
}
// This is called for every layer with a provider configured with a matching name at startup time. This should return your provider type with any initialization logic, the simplest case is just passing your config struct into your provider struct like shown here.
// Everything the provider is given at construction arrives in the deps struct: deps.ClientConfig, deps.ErrorMessages, deps.LayerGroup and deps.Datastores.
func (s SampleRegistration) Initialize(cfgAny any, deps layer.ProviderDeps) (layer.Provider, error) {
config := cfgAny.(SampleConfig) //This will always be a mutated version of what's returned from InitializeConfig
return &Sample{config}, nil //An error returned here will prevent startup
}
func (t Sample) PreAuth(ctx context.Context, providerContext layer.ProviderContext) (layer.ProviderContext, error) {
return providerContext, nil
}
func (t Sample) GenerateTile(ctx context.Context, providerContext layer.ProviderContext, tileRequest pkg.TileRequest) (*pkg.Image, error) {
return nil, errors.New("not implemented")
}A provider that can describe its own tiles can also implement the optional layer.MetadataProvider interface, returning a layer.Description. Providers describe, they never restrict. See layer metadata for how the layer uses it. A provider that nests other providers should implement layer.Parent so its children are described and closed along with it.
From here, implementing the provider is the same as implementing a Custom provider. The other entities can be specified in the same way.
A complete project implementing every entity type, along with a wrapper executable that tests its configuration before serving it, can be found in examples/extension.
The pkg package contains everything available for writing your own entities:
| Package | Contents |
|---|---|
|
|
|
The configuration structs, including |
|
One package per entity type holding its interface, the |
|
The optional |
Apart from cmd.Execute and cmd.ExecuteArgs, everything else lives under internal/ and cannot be imported.