Documentation
¶
Overview ¶
Package gozip provides a high-performance, concurrency-safe, and feature-rich implementation of the ZIP archive format.
It is designed as a robust alternative to the standard library's archive/zip, specifically built for high-load applications, security-conscious environments, and scenarios requiring legacy compatibility.
Key Features ¶
1. Concurrency: Unlike the standard library, gozip supports parallel compression (WriteToParallel) and parallel extraction (ExtractParallel), scaling linearly with CPU cores.
2. Security: Native support for WinZip AES-256 encryption (reading and writing) and built-in "Zip Slip" protection during extraction to prevent directory traversal attacks.
3. Context Awareness: All long-running operations support context.Context for cancellation and timeout management, making it ideal for HTTP handlers and background jobs.
4. Compatibility: Handles Zip64 (files > 4GB), NTFS timestamps, Unix permissions, and legacy DOS encodings (CP437, CP866) automatically.
5. File System Interface: The archive can be accessed as a read-only filesystem using the fs.FS interface. This allows seamless integration with Go's standard filesystem APIs, such as io/fs and path/filepath. Example:
archive := gozip.NewZip() // ... add files to the archive ... fsys := archive.FS() data, _ := fs.ReadFile(fsys, "file.txt")
Basic Usage ¶
Creating an archive sequentially:
archive := gozip.NewZip()
archive.AddFile("file.txt")
archive.AddDir("images/", WithCompression(gozip.Deflate, gozip.DeflateMaximum))
f, _ := os.Create("output.zip")
archive.WriteTo(f)
Creating an archive in parallel (faster for multiple files):
// compress using 8 workers archive.WriteToParallel(f, 8)
Modifying an existing archive:
archive := gozip.NewZip()
src, _ := os.Open("old.zip")
archive.LoadFromFile(src)
// 1. Remove obsolete files
archive.Remove("logs/obsolete.log")
// 2. Replace a file
file, _ := archive.File("data/config.json")
archive.Remove(file.Name())
// Modify file data by safely reading it from source archive
archive.AddLazy("data/config.json", func() (io.ReadCloser, error) {
rc, _ := file.Open()
defer rc.Close()
// Modify original data ...
return io.NopCloser(bytes.NewReader(processedData)), nil
})
// 3. Rename entries
archive.Rename("dir/old", "new") // -> dir/new
// Save changes to a new writer
dest, _ := os.Create("new.zip")
archive.WriteTo(dest)
// Close source after the work is done
src.Close()
Index ¶
- Constants
- Variables
- func DecodeCP437(s string) string
- func DecodeIBM866(s string) string
- type AddOption
- func WithCompression(c CompressionMethod, lvl int) AddOption
- func WithConfig(c FileConfig) AddOption
- func WithEncryption(e EncryptionMethod, pwd string) AddOption
- func WithMode(mode fs.FileMode) AddOption
- func WithName(name string) AddOption
- func WithPassword(pwd string) AddOption
- func WithPath(p string) AddOption
- type CompressionMethod
- type Compressor
- type CompressorFactory
- type Decompressor
- type DeflateCompressor
- type DeflateDecompressor
- type EncryptionMethod
- type ExtractOption
- type File
- func (f *File) CRC32() uint32
- func (f *File) CompressedSize() int64
- func (f *File) Config() FileConfig
- func (f *File) FsTime() (mtime, atime, ctime time.Time)
- func (f *File) GetExtraField(tag uint16) []byte
- func (f *File) HasExtraField(tag uint16) bool
- func (f *File) HostSystem() sys.HostSystem
- func (f *File) IsDir() bool
- func (f *File) ModTime() time.Time
- func (f *File) Mode() fs.FileMode
- func (f *File) Name() string
- func (f *File) Open() (io.ReadCloser, error)
- func (f *File) OpenWithPassword(pwd string) (io.ReadCloser, error)
- func (f *File) RequiresZip64() bool
- func (f *File) SetConfig(config FileConfig)
- func (f *File) SetExtraField(tag uint16, data []byte) error
- func (f *File) SetOpenFunc(openFunc func() (io.ReadCloser, error))
- func (f *File) UncompressedSize() int64
- type FileConfig
- type FileSortStrategy
- type StoredCompressor
- type StoredDecompressor
- type TextDecoder
- type Zip
- func (z *Zip) AddBytes(data []byte, filename string, options ...AddOption) error
- func (z *Zip) AddDir(path string, options ...AddOption) error
- func (z *Zip) AddFS(fileSystem fs.FS, options ...AddOption) error
- func (z *Zip) AddFile(path string, options ...AddOption) error
- func (z *Zip) AddLazy(name string, openFunc func() (io.ReadCloser, error), options ...AddOption) error
- func (z *Zip) AddOSFile(f *os.File, options ...AddOption) error
- func (z *Zip) AddReader(r io.Reader, filename string, size int64, options ...AddOption) error
- func (z *Zip) AddString(content string, filename string, options ...AddOption) error
- func (z *Zip) Exists(name string) bool
- func (z *Zip) Extract(path string, options ...ExtractOption) error
- func (z *Zip) ExtractParallel(path string, workers int, options ...ExtractOption) error
- func (z *Zip) ExtractParallelWithContext(ctx context.Context, path string, workers int, options ...ExtractOption) error
- func (z *Zip) ExtractWithContext(ctx context.Context, path string, options ...ExtractOption) error
- func (z *Zip) FS() fs.FS
- func (z *Zip) File(name string) (*File, error)
- func (z *Zip) Files() []*File
- func (z *Zip) Find(pattern string) ([]*File, error)
- func (z *Zip) Glob(pattern string) ([]*File, error)
- func (z *Zip) Load(src io.ReaderAt, size int64) error
- func (z *Zip) LoadFromFile(f *os.File) error
- func (z *Zip) LoadFromFileWithContext(ctx context.Context, f *os.File) error
- func (z *Zip) LoadWithContext(ctx context.Context, src io.ReaderAt, size int64) error
- func (z *Zip) Mkdir(name string, options ...AddOption) error
- func (z *Zip) Move(old, new string) error
- func (z *Zip) OpenFile(name string) (io.ReadCloser, error)
- func (z *Zip) RegisterCompressor(method CompressionMethod, factory CompressorFactory)
- func (z *Zip) RegisterDecompressor(method CompressionMethod, d Decompressor)
- func (z *Zip) Remove(name string) error
- func (z *Zip) Rename(old, new string) error
- func (z *Zip) SetConfig(c ZipConfig)
- func (z *Zip) WriteTo(dest io.Writer) (int64, error)
- func (z *Zip) WriteToParallel(dest io.Writer, maxWorkers int) (int64, error)
- func (z *Zip) WriteToParallelWithContext(ctx context.Context, dest io.Writer, maxWorkers int) (int64, error)
- func (z *Zip) WriteToWithContext(ctx context.Context, dest io.Writer) (int64, error)
- type ZipConfig
Constants ¶
const ( DeflateNormal = flate.DefaultCompression // -1 DeflateMaximum = flate.BestCompression // 9 DeflateFast = 3 DeflateSuperFast = flate.BestSpeed // 1 (Same as Fast in stdlib) DeflateStore = flate.NoCompression // 0 )
Compression levels for DEFLATE algorithm.
const ( // LatestZipVersion represents the maximum ZIP specification version supported // by this implementation. Version 63 corresponds to ZIP 6.3 specification. LatestZipVersion uint16 = 63 // Zip64ExtraFieldTag identifies the extra field that contains 64-bit size // and offset information for files exceeding 4GB limits. Zip64ExtraFieldTag uint16 = 0x0001 // NTFSFieldTag identifies the extra field that stores high-precision // NTFS file timestamps with 100-nanosecond resolution. NTFSFieldTag uint16 = 0x000A // AESEncryptionTag identifies the extra field for WinZip AES encryption metadata, // including encryption strength and actual compression method. AESEncryptionTag uint16 = 0x9901 )
Constants defining ZIP format structure and special tag values
const SizeUnknown int64 = -1
SizeUnknown is a sentinel value used when the uncompressed size of a file cannot be determined before writing (e.g., streaming from io.Reader).
Variables ¶
var ( // ErrFormat is returned when the input is not a valid ZIP archive. ErrFormat = errors.New("zip: not a valid zip file") // ErrFileEntry is returned when an invalid argument is passed to File creation. ErrFileEntry = errors.New("zip: not a valid file entry") // ErrAlgorithm is returned when a compression algorithm is not supported. ErrAlgorithm = errors.New("unsupported compression algorithm") // ErrEncryption is returned when an encryption method is not supported. ErrEncryption = errors.New("unsupported encryption method") // ErrPasswordMismatch is returned when the provided password does not match // or when a password is required but not provided. ErrPasswordMismatch = errors.New("zip: invalid password") // ErrChecksum is returned when reading a file checksum does not match. ErrChecksum = errors.New("zip: checksum error") // ErrSizeMismatch is returned when the uncompressed size does not match the header. ErrSizeMismatch = errors.New("zip: uncompressed size mismatch") // ErrFileNotFound is returned when the requested file is not found in the archive. // It wraps fs.ErrNotExist so it can be checked with os.IsNotExist. ErrFileNotFound = fmt.Errorf("zip: file not found: %w", fs.ErrNotExist) // ErrInsecurePath is returned when a file path is invalid or attempts directory traversal (Zip Slip). ErrInsecurePath = errors.New("zip: insecure file path") // ErrDuplicateEntry is returned when attempting to add a file with a name that already exists. ErrDuplicateEntry = errors.New("zip: duplicate file name") // ErrFilenameTooLong is returned when a filename exceeds 65535 bytes. ErrFilenameTooLong = errors.New("zip: filename too long") // ErrCommentTooLong is returned when a file comment exceeds 65535 bytes. ErrCommentTooLong = errors.New("zip: comment too long") // ErrExtraFieldTooLong is returned when the total size of extra fields exceeds 65535 bytes. ErrExtraFieldTooLong = errors.New("zip: extra field too long") )
Functions ¶
func DecodeCP437 ¶ added in v1.2.0
DecodeCP437 converts CP437 (US DOS) string to UTF-8. This is the default fallback for ZIP archives.
func DecodeIBM866 ¶
DecodeIBM866 converts CP866 (Cyrillic DOS) string to UTF-8.
Types ¶
type AddOption ¶
type AddOption func(f *File)
AddOption is a functional option for configuring file entries during addition.
func WithCompression ¶
func WithCompression(c CompressionMethod, lvl int) AddOption
WithCompression sets the compression method and level for a regular file. Ignored for directories.
func WithConfig ¶
func WithConfig(c FileConfig) AddOption
WithConfig applies a complete FileConfig, overwriting existing settings.
func WithEncryption ¶
func WithEncryption(e EncryptionMethod, pwd string) AddOption
WithEncryption sets the encryption method and password for a regular file. Ignored for directories.
func WithMode ¶
WithMode sets the Unix-style permission bits. This affects the external attributes field in the ZIP header.
func WithName ¶
WithName overrides the destination filename within the archive. The name is automatically normalized to use forward slashes.
func WithPassword ¶ added in v1.3.0
WithPassword sets the encryption password for a specific file. If no encryption method is specified, it defaults to AES256. Ignored for directories.
type CompressionMethod ¶
type CompressionMethod uint16
CompressionMethod represents the compression algorithm used for a file in the ZIP archive
const ( Store CompressionMethod = 0 // No compression - file stored as-is Deflate CompressionMethod = 8 // DEFLATE compression (most common) Deflate64 CompressionMethod = 9 // DEFLATE64(tm) - Not supported natively BZIP2 CompressionMethod = 12 // BZIP2 - Not supported natively LZMA CompressionMethod = 14 // LZMA - Not supported natively ZStandard CompressionMethod = 93 // Zstandard - Not supported natively )
Supported compression methods according to ZIP specification. Note: This library natively supports Store (0) and Deflate (8). Other methods require registering custom compressors via RegisterCompressor.
type Compressor ¶
type Compressor interface {
// Compress reads from src and writes compressed data to dest.
// Returns the number of uncompressed bytes read.
Compress(src io.Reader, dest io.Writer) (int64, error)
}
Compressor transforms raw data into compressed data.
type CompressorFactory ¶ added in v1.2.0
type CompressorFactory func(level int) Compressor
CompressorFactory creates a Compressor instance for a specific compression level. The level parameter is typically 0-9, but interpretations vary by algorithm. Implementations should normalize invalid levels to defaults.
type Decompressor ¶
type Decompressor interface {
// Decompress returns a stream of uncompressed data.
Decompress(src io.Reader) (io.ReadCloser, error)
}
Decompressor transforms compressed data back into raw data.
type DeflateCompressor ¶
type DeflateCompressor struct {
// contains filtered or unexported fields
}
DeflateCompressor implements DEFLATE compression with memory pooling.
func NewDeflateCompressor ¶
func NewDeflateCompressor(level int) *DeflateCompressor
NewDeflateCompressor creates a reusable compressor for a specific level.
type DeflateDecompressor ¶
type DeflateDecompressor struct{}
DeflateDecompressor implements the "Deflate" method.
func (*DeflateDecompressor) Decompress ¶
func (dd *DeflateDecompressor) Decompress(src io.Reader) (io.ReadCloser, error)
type EncryptionMethod ¶
type EncryptionMethod uint16
EncryptionMethod represents the encryption algorithm used for file protection.
const ( NotEncrypted EncryptionMethod = 0 // No encryption - file stored in plaintext ZipCrypto EncryptionMethod = 1 // Legacy encryption. Vulnerable to brute force attacks AES256 EncryptionMethod = 2 // Modern AES256 encryption )
Supported encryption methods
type ExtractOption ¶ added in v1.1.0
ExtractOption configures the extraction process (filtering).
func FromDir ¶ added in v1.1.0
func FromDir(path string) ExtractOption
FromDir restricts extraction to files nested under the specified path.
func WithFiles ¶ added in v1.1.0
func WithFiles(files []*File) ExtractOption
WithFiles filters the extraction to only the specific files provided.
func WithoutDir ¶ added in v1.1.0
func WithoutDir(path string) ExtractOption
WithoutDir excludes a directory and its contents from extraction.
type File ¶
type File struct {
// contains filtered or unexported fields
}
File represents a file entry within a ZIP archive, encapsulating both metadata and content access mechanisms. Each File object corresponds to one entry in the ZIP central directory and can represent either a regular file or a directory.
func SortFilesOptimized ¶
func SortFilesOptimized(files []*File, strategy FileSortStrategy) []*File
SortFilesOptimized returns a sorted slice of files according to the strategy. Returns a new slice; the original slice is not modified.
func (*File) CompressedSize ¶
CompressedSize returns the size of the compressed data within the archive.
func (*File) Config ¶ added in v1.1.0
func (f *File) Config() FileConfig
Config returns archive file entry configuration.
func (*File) FsTime ¶ added in v1.1.0
FsTime returns the file timestamps (Modification, Access, Creation) if available.
func (*File) GetExtraField ¶
GetExtraField retrieves the raw bytes of an extra field by its tag ID.
func (*File) HasExtraField ¶
HasExtraField checks whether an extra field with the specified tag exists.
func (*File) HostSystem ¶ added in v1.1.0
func (f *File) HostSystem() sys.HostSystem
HostSystem returns the system file was created in.
func (*File) Open ¶
func (f *File) Open() (io.ReadCloser, error)
Open returns a ReadCloser for reading the original, uncompressed file content.
func (*File) OpenWithPassword ¶ added in v1.3.0
func (f *File) OpenWithPassword(pwd string) (io.ReadCloser, error)
Open returns a ReadCloser object for reading the original, uncompressed file content using the specified password.
func (*File) RequiresZip64 ¶
RequiresZip64 determines whether this file requires ZIP64 format extensions.
func (*File) SetConfig ¶
func (f *File) SetConfig(config FileConfig)
SetConfig applies a FileConfig to this file, overriding individual properties.
func (*File) SetExtraField ¶
SetExtraField adds or replaces an extra field entry for this file. Returns an error if adding the field would exceed the maximum extra field length.
func (*File) SetOpenFunc ¶
func (f *File) SetOpenFunc(openFunc func() (io.ReadCloser, error))
SetOpenFunc replaces the internal function used to open the file's content. Note that internal file sizes will be updated only after the archive is written.
func (*File) UncompressedSize ¶
UncompressedSize returns the size of the original file content before compression.
type FileConfig ¶
type FileConfig struct {
// CompressionMethod overrides the global default.
CompressionMethod CompressionMethod
// CompressionLevel overrides the global default.
CompressionLevel int
// EncryptionMethod overrides the global default.
EncryptionMethod EncryptionMethod
// Password overrides the global archive password for this file.
Password string
// Comment is a file-specific comment (max 65535 bytes).
Comment string
}
FileConfig defines configuration specific to a single archive entry. It overrides the global ZipConfig.
type FileSortStrategy ¶
type FileSortStrategy int
FileSortStrategy defines the order in which files are written to the archive. Choosing the right strategy can optimize writing speed (CPU parallelism) or archive structure (ZIP64 overhead).
const ( SortDefault FileSortStrategy = iota SortLargeFilesLast // Large files (>=4GB) at end SortLargeFilesFirst // Large files (>=4GB) at start SortSizeAscending // Smallest first SortSizeDescending // Largest first SortZIP64Optimized // Buckets: <10MB, <4GB, >=4GB (each sorted Asc) SortAlphabetical // A-Z by filename )
type StoredCompressor ¶
type StoredCompressor struct{}
StoredCompressor implements no compression (STORE method).
type StoredDecompressor ¶
type StoredDecompressor struct{}
StoredDecompressor implements the "Store" method (no compression).
func (*StoredDecompressor) Decompress ¶
func (sd *StoredDecompressor) Decompress(src io.Reader) (io.ReadCloser, error)
type TextDecoder ¶
TextDecoder is a function that converts raw binary string (interpreted as specific encoding) into UTF-8.
type Zip ¶
type Zip struct {
// contains filtered or unexported fields
}
Zip represents an in-memory ZIP archive manager. It is concurrency-safe and supports streaming, random access, and parallel operations.
func NewZip ¶
func NewZip() *Zip
NewZip creates a ready-to-use empty ZIP archive. Default support includes Store (No Compression) and Deflate.
func (*Zip) AddDir ¶ added in v1.3.0
AddDir recursively adds a local directory and its contents to the archive. Returns a combined error if any files fail to add (Best Effort). Symlinks aren't followed.
func (*Zip) AddFS ¶ added in v1.3.0
AddFS adds files from an fs.FS (e.g., embed.FS, os.DirFS) to the archive. It recursively walks the file system and adds all entries.
func (*Zip) AddFile ¶
AddFile adds a file from the local filesystem to the archive. Opens, reads, and closes the file automatically. Symlinks aren't followed.
func (*Zip) AddLazy ¶ added in v1.3.0
func (z *Zip) AddLazy(name string, openFunc func() (io.ReadCloser, error), options ...AddOption) error
AddLazy adds a new file with unknown size and openFunc.
func (*Zip) AddOSFile ¶ added in v1.2.0
AddOSFile adds an open *os.File to the archive. Uses native OS metadata. The whole file content will be added using io.SectionReader.
func (*Zip) AddReader ¶
AddReader streams content from an io.Reader into the archive. Use SizeUnknown for 'size' if the length is not known ahead of time.
func (*Zip) Exists ¶
Exists checks if a file or directory exists in the archive. Supports both exact matches and directory prefixes. Thread-safe.
func (*Zip) Extract ¶
func (z *Zip) Extract(path string, options ...ExtractOption) error
Extract extracts files to the destination directory. Includes Zip Slip protection to ensure files stay within the target path. If the file is encrypted and the password does not match, it will be reset so that it can be replaced with a new zip config password in the next attempt.
func (*Zip) ExtractParallel ¶
func (z *Zip) ExtractParallel(path string, workers int, options ...ExtractOption) error
ExtractParallel extracts files using multiple workers. This is IO-bound optimized. Missing directories are created automatically.
func (*Zip) ExtractParallelWithContext ¶ added in v1.1.0
func (z *Zip) ExtractParallelWithContext(ctx context.Context, path string, workers int, options ...ExtractOption) error
ExtractParallelWithContext extracts files concurrently with context support.
func (*Zip) ExtractWithContext ¶ added in v1.1.0
ExtractWithContext extracts files with context support.
func (*Zip) File ¶ added in v1.2.0
File returns the entry matching the given name. Name is case-sensitive and normalized to forward slashes. Returns ErrFileNotFound if no exact match is found.
func (*Zip) Find ¶ added in v1.2.0
Find searches for files matching the pattern in all directories. Unlike Glob, the pattern "*" matches "/" characters. Example: Find("*.log") matches "error.log" AND "var/logs/access.log".
func (*Zip) Glob ¶ added in v1.2.0
Glob returns all files whose names match the specified shell pattern. Pattern syntax is identical to path.Match.
func (*Zip) Load ¶ added in v1.2.0
Load parses an existing ZIP archive from the reader and appends its entries to this struct. It does not load file contents into memory, only the directory structure. Loaded files will have the current config password.
func (*Zip) LoadFromFile ¶ added in v1.2.0
LoadFromFile parses a ZIP from a local os.File.
func (*Zip) LoadFromFileWithContext ¶ added in v1.3.0
LoadFromFile parses a ZIP from a local os.File with context support.
func (*Zip) LoadWithContext ¶ added in v1.2.0
LoadWithContext parses an archive with context support.
func (*Zip) Mkdir ¶ added in v1.2.0
Mkdir creates an explicit directory entry in the archive. Note: Directories are created implicitly by file paths; this is used for empty directories or specific metadata.
func (*Zip) Move ¶ added in v1.1.0
Move changes the directory location of a file while preserving its base name. e.g., "docs/file.txt", "backup/docs" -> "backup/docs/file.txt". Directories are moved recursively. Missing parent directories are created automatically.
func (*Zip) OpenFile ¶
func (z *Zip) OpenFile(name string) (io.ReadCloser, error)
OpenFile returns a ReadCloser for the named file within the archive. Returns ErrFileNotFound if not found.
func (*Zip) RegisterCompressor ¶
func (z *Zip) RegisterCompressor(method CompressionMethod, factory CompressorFactory)
RegisterCompressor registers a factory function for a specific compression method. The factory will be called when a file requires this method at a specific level.
func (*Zip) RegisterDecompressor ¶
func (z *Zip) RegisterDecompressor(method CompressionMethod, d Decompressor)
RegisterDecompressor adds support for reading a custom compression method.
func (*Zip) Remove ¶ added in v1.3.0
Remove removes a file or directory from the archive. Empty string or "." means to delete all files. If the target is a directory, it recursively removes all files and subdirectories inside it. Returns an error if no files were deleted.
func (*Zip) Rename ¶ added in v1.1.0
Rename changes a file's name while preserving its directory location. e.g., "logs/old.txt", "add/new.txt" -> "logs/add/new.txt". If the target is a directory, all children are recursively renamed.
func (*Zip) WriteTo ¶ added in v1.2.0
WriteTo serializes the ZIP archive to the specified io.Writer. Returns the total number of bytes written to the writer. This is a sequential operation and will finalize the archive structure.
func (*Zip) WriteToParallel ¶ added in v1.2.0
WriteToParallel writes the archive using concurrent workers for compression. Best used when the archive contains many large files that benefit from parallel CPU usage. Returns the total bytes written to dest.
type ZipConfig ¶
type ZipConfig struct {
// CompressionMethod is the default algorithm for new files.
CompressionMethod CompressionMethod
// CompressionLevel controls the speed vs size trade-off (0-9).
// 0 = Store (no compression), 9 = Best compression.
CompressionLevel int
// EncryptionMethod is the default encryption algorithm.
// Recommended: AES256.
EncryptionMethod EncryptionMethod
// Password is the default credentials for encrypting the archive.
Password string
// Comment is the archive-level comment (max 65535 bytes).
Comment string
// FileSortStrategy defines the order of files in the written archive.
FileSortStrategy FileSortStrategy
// TextEncoding handles filename decoding for non-UTF8 legacy archives.
// Default: CP437 (IBM PC).
TextEncoding TextDecoder
// OnFileProcessed is a callback triggered after a file is successfully
// written, read, or extracted.
// WARNING: In parallel operations, this is called concurrently.
OnFileProcessed func(*File, error)
}
ZipConfig defines global configuration parameters for the archive. These settings apply to the entire archive but can be overridden per-file using FileConfig options.