pmtiles

package
v0.0.0-...-19f7fc4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2025 License: BSD-3-Clause Imports: 54 Imported by: 0

Documentation

Index

Constants

View Source
const (
	UnknownCompression Compression = 0
	NoCompression                  = 1
	Gzip                           = 2
	Brotli                         = 3
	Zstd                           = 4
)
View Source
const (
	UnknownTileType TileType = 0
	Mvt                      = 1
	Png                      = 2
	Jpeg                     = 3
	Webp                     = 4
	Avif                     = 5
)
View Source
const HeaderV3LenBytes = 127

HeaderV3LenBytes is the fixed-size binary header size.

Variables

This section is empty.

Functions

func BboxRegion

func BboxRegion(bbox string) (orb.MultiPolygon, error)

BboxRegion parses a bbox string into an orb.MultiPolygon region.

func Cluster

func Cluster(logger *log.Logger, InputPMTiles string, deduplicate bool) error

func Convert

func Convert(logger *log.Logger, input string, output string, deduplicate bool, tmpfile *os.File) error

Convert an existing archive on disk to a new PMTiles specification version 3 archive.

func CreateTileJSON

func CreateTileJSON(header HeaderV3, metadataBytes []byte, tileURL string) ([]byte, error)

CreateTileJSON returns TileJSON from an archive header+metadata and a given public tileURL.

func DeserializeMetadata

func DeserializeMetadata(reader io.Reader, compression Compression) (map[string]interface{}, error)

func DeserializeMetadataBytes

func DeserializeMetadataBytes(reader io.Reader, compression Compression) ([]byte, error)

func Edit

func Edit(_ *log.Logger, inputArchive string, newHeaderJSONFile string, newMetadataFile string) error

Edit parts of the header or metadata. works in-place if only the header is modified.

func Extract

func Extract(_ *log.Logger, bucketURL string, key string, minzoom int8, maxzoom int8, regionFile string, bbox string, output string, downloadThreads int, overfetch float32, dryRun bool) error

Extract a smaller archive from local or remote archive. 1. Get the root directory (check that it is clustered) 2. Turn the input geometry into a relevance bitmap (using min(maxzoom, headermaxzoom)) 3. Get all relevant level 1 directories (if any) 4. Get all relevant level 2 directories (usually none) 5. With the existing directory + relevance bitmap, construct

  • a new total directory (root + leaf directories)
  • a sorted slice of byte ranges in the old file required

6. Merge requested ranges using an overfetch parametter 7. write the modified header 8. write the root directory. 9. get and write the metadata. 10. write the leaf directories (if any) 11. Get all tiles, and write directly to the output.

func IDToZxy

func IDToZxy(i uint64) (uint8, uint32, uint32)

IDToZxy converts a Hilbert TileID to (Z,X,Y) tile coordinates.

func IterateEntries

func IterateEntries(header HeaderV3, fetch func(uint64, uint64) ([]byte, error), operation func(EntryV3)) error

func Makesync

func Makesync(logger *log.Logger, cliVersion string, fileName string, blockSizeKb int) error

func MergeRanges

func MergeRanges(ranges []srcDstRange, overfetch float32) (*list.List, uint64)

MergeRanges takes a slice of SrcDstRanges, that: * is non-contiguous, and is sorted by DstOffset * an Overfetch parameter

  • overfetch = 0.2 means we can request an extra 20%
  • overfetch = 1.00 means we can double our total transfer size

Return a list of OverfetchRanges

Each OverfetchRange is one or more input ranges
input ranges are merged in order of smallest byte distance to next range
until the overfetch budget is consumed.
The list is sorted by Length

func NewCors

func NewCors(corsOrigins string) *cors.Cors

func NormalizeBucketKey

func NormalizeBucketKey(bucket string, prefix string, key string) (string, string, error)

func ParentID

func ParentID(i uint64) uint64

ParentID efficiently finds a parent Hilbert TileID without converting to (Z,X,Y).

func RelevantEntries

func RelevantEntries(bitmap *roaring64.Bitmap, maxzoom uint8, dir []EntryV3) ([]EntryV3, []EntryV3)

RelevantEntries finds the intersection of a bitmap and a directory return sorted slice of entries, and slice of all leaf entries any runlengths > 1 will be "trimmed" to the relevance bitmap

func SerializeEntries

func SerializeEntries(entries []EntryV3, compression Compression) []byte

func SerializeHeader

func SerializeHeader(header HeaderV3) []byte

func SerializeMetadata

func SerializeMetadata(metadata map[string]interface{}, compression Compression) ([]byte, error)

func SetBuildInfo

func SetBuildInfo(version, commit, date string)

SetBuildInfo initializes static metrics with pmtiles version, git hash, and build time

func Show

func Show(_ *log.Logger, output io.Writer, bucketURL string, key string, showHeaderJsonOnly bool, showMetadataOnly bool, showTilejson bool, publicURL string, showTile bool, z int, x int, y int) error

Show prints detailed information about an archive.

func Sync

func Sync(logger *log.Logger, oldVersion string, newVersion string, dryRun bool) error

func UnmarshalRegion

func UnmarshalRegion(data []byte) (orb.MultiPolygon, error)

UnmarshalRegion parses JSON bytes into an orb.MultiPolygon region.

func Upload

func Upload(_ *log.Logger, InputPMTiles string, bucket string, RemotePMTiles string, maxConcurrency int) error

Upload a pmtiles archive to a bucket.

func Verify

func Verify(_ *log.Logger, file string) error

Verify that an archive's header statistics are correct, and that tiles are propertly ordered if clustered=true.

func ZxyToID

func ZxyToID(z uint8, x uint32, y uint32) uint64

ZxyToID converts (Z,X,Y) tile coordinates to a Hilbert TileID.

Types

type Bucket

type Bucket interface {
	Close() error
	NewRangeReader(ctx context.Context, key string, offset int64, length int64) (io.ReadCloser, error)
	NewRangeReaderEtag(ctx context.Context, key string, offset int64, length int64, etag string) (io.ReadCloser, string, int, error)
}

Bucket is an abstration over a gocloud or plain HTTP bucket.

func OpenBucket

func OpenBucket(ctx context.Context, bucketURL string, bucketPrefix string) (Bucket, error)

type BucketAdapter

type BucketAdapter struct {
	Bucket *blob.Bucket
}

func (BucketAdapter) Close

func (ba BucketAdapter) Close() error

func (BucketAdapter) NewRangeReader

func (ba BucketAdapter) NewRangeReader(ctx context.Context, key string, offset, length int64) (io.ReadCloser, error)

func (BucketAdapter) NewRangeReaderEtag

func (ba BucketAdapter) NewRangeReaderEtag(ctx context.Context, key string, offset, length int64, etag string) (io.ReadCloser, string, int, error)

type Compression

type Compression uint8

Compression is the compression algorithm applied to individual tiles (or none)

type EntryV3

type EntryV3 struct {
	TileID    uint64
	Offset    uint64
	Length    uint32
	RunLength uint32
}

EntryV3 is an entry in a PMTiles spec version 3 directory.

func DeserializeEntries

func DeserializeEntries(data *bytes.Buffer, compression Compression) []EntryV3

func FindTile

func FindTile(entries []EntryV3, tileID uint64) (EntryV3, bool)

type FileBucket

type FileBucket struct {
	// contains filtered or unexported fields
}

FileBucket is a bucket backed by a directory on disk

func NewFileBucket

func NewFileBucket(path string) *FileBucket

NewFileBucket initializes a FileBucket and returns a new instance

func (FileBucket) Close

func (b FileBucket) Close() error

func (FileBucket) NewRangeReader

func (b FileBucket) NewRangeReader(ctx context.Context, key string, offset, length int64) (io.ReadCloser, error)

func (FileBucket) NewRangeReaderEtag

func (b FileBucket) NewRangeReaderEtag(_ context.Context, key string, offset, length int64, etag string) (io.ReadCloser, string, int, error)

type HTTPBucket

type HTTPBucket struct {
	// contains filtered or unexported fields
}

func (HTTPBucket) Close

func (b HTTPBucket) Close() error

func (HTTPBucket) NewRangeReader

func (b HTTPBucket) NewRangeReader(ctx context.Context, key string, offset, length int64) (io.ReadCloser, error)

func (HTTPBucket) NewRangeReaderEtag

func (b HTTPBucket) NewRangeReaderEtag(ctx context.Context, key string, offset, length int64, etag string) (io.ReadCloser, string, int, error)

type HTTPClient

type HTTPClient interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPClient is an interface that lets you swap out the default client with a mock one in tests

type HeaderJson

type HeaderJson struct {
	TileCompression string    `json:"tile_compression"`
	TileType        string    `json:"tile_type"`
	MinZoom         int       `json:"minzoom"`
	MaxZoom         int       `json:"maxzoom"`
	Bounds          []float64 `json:"bounds"`
	Center          []float64 `json:"center"`
}

HeaderJson is a human-readable representation of parts of the binary header that may need to be manually edited. Omitted parts are the responsibility of the generator program and not editable. The formatting is aligned with the TileJSON / MBTiles specification.

type HeaderV3

type HeaderV3 struct {
	SpecVersion         uint8
	RootOffset          uint64
	RootLength          uint64
	MetadataOffset      uint64
	MetadataLength      uint64
	LeafDirectoryOffset uint64
	LeafDirectoryLength uint64
	TileDataOffset      uint64
	TileDataLength      uint64
	AddressedTilesCount uint64
	TileEntriesCount    uint64
	TileContentsCount   uint64
	Clustered           bool
	InternalCompression Compression
	TileCompression     Compression
	TileType            TileType
	MinZoom             uint8
	MaxZoom             uint8
	MinLonE7            int32
	MinLatE7            int32
	MaxLonE7            int32
	MaxLatE7            int32
	CenterZoom          uint8
	CenterLonE7         int32
	CenterLatE7         int32
}

HeaderV3 is a binary header for PMTiles specification version 3.

func DeserializeHeader

func DeserializeHeader(d []byte) (HeaderV3, error)

type RefreshRequiredError

type RefreshRequiredError struct {
	StatusCode int
}

RefreshRequiredError is an error that indicates the etag has chanced on the remote file

func (*RefreshRequiredError) Error

func (m *RefreshRequiredError) Error() string

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is an HTTP server for tiles and metadata.

func NewServer

func NewServer(bucketURL string, prefix string, logger *log.Logger, cacheSize int, publicURL string) (*Server, error)

NewServer creates a new pmtiles HTTP server.

func NewServerWithBucket

func NewServerWithBucket(bucket Bucket, _ string, logger *log.Logger, cacheSize int, publicURL string) (*Server, error)

NewServerWithBucket creates a new HTTP server for a gocloud Bucket.

func (*Server) Get

func (server *Server) Get(ctx context.Context, path string) (int, map[string]string, []byte)

Get a response for the given path. Return status code, HTTP headers, and body.

func (*Server) ServeHTTP

func (server *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) int

Serve an HTTP response from the archive

func (*Server) Start

func (server *Server) Start()

Start the server HTTP listener.

type TileType

type TileType uint8

TileType is the format of individual tile contents in the archive.

type Zxy

type Zxy struct {
	Z uint8
	X uint32
	Y uint32
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL