Public Zone Database

The Public Zone Database (ZoneDB) is a free, open-source database containing a list and associated metadata of public DNS zones (domain name extensions). It attempts to be exhaustive, including current, retired, and withdrawn top-level domains and subdomains.

The intended use case is programs that interface with the public domain name system, including registry and registrar services. The data overlaps with and shares test cases with the Public Suffix List maintained by the Mozilla Foundation. The source data was originally derived from the internal zone database in use at Domainr.

Structure

The database consists of a list of zones (zones.txt) and associated metadata in JSON format.

CLI

The zonedb CLI is fully documented. Run go run cmd/zonedb/main.go -h to see its arguments.

zones.txt

The zones.txt file is a UTF-8 encoded text file containing a list of IDN & lower-case domain names, one per line, followed by a newline (\n) character. The domain names in zones.txt are sorted according to the following rules:

1. Number of labels (top-level domains, followed by second- and third-level domains)
2. Parent domain (e.g. uk)
3. Subdomain (e.g. co.uk)

Metadata

Each domain with associated metadata will have a JSON file in the metadata directory.

RDAP

When adding an RDAP server URL, only the base URL is needed. e.g. https://rdap.sidn.nl/

Updates

A GitHub Actions workflow updates ZoneDB each night.

If a new SLD or third-level domain needs to be added to zones.txt, follow these steps locally:

1. Add the new zone to the bottom of the zones.txt file.
2. Run make normalize to normalize the data changes.
3. Run go run cmd/zonedb/main.go -update -w -zones {new zone} to update its metadata.
4. Create a pull request for the changes, and confirm the tests are passing.

JSON files

If an update to one of the {zone}.json files is needed, do this locally:

1. make normalize # tidies the json files, and runs go generate
2. commit the changes

Implementations

Go

go get github.com/zonedb/zonedb

This repository contains a reference implementation of the database written in Go. Unlike the source data, which is in Unicode, the Go implementation represents domain names in ASCII IDNA form for interop with existing libraries.

Go Support

ZoneDB supports a minimum of 3 Go releases. As of October 2023, this includes Go 1.21, 1.20, and 1.19.

Contributing

Fork this repository, make changes, and send a pull request. Before submitting a PR, run make normalize to normalize any changes. The data is structured to minimize diff size.

zonedb build tool

This database is generated and validated using the zonedb tool in this directory. To install, make sure you have a working Go installation (1.4+) and run this command:

go get -u github.com/zonedb/zonedb/cmd/zonedb

You can also run the zonedb tool directly: go run cmd/zonedb/main.go

Example commands

List all zones that have wildcarded DNS:

zonedb -list-wildcards

List all zones tagged geo:

zonedb -tags geo

List a given zone's tags:

zonedb -zones capetown -list-tags

Add a tag to multiple zones (and write the output):

zonedb -zones capetown,durban,joburg -add-tags city -w

Remove a tag from a zone (and write the output):

zonedb -zones la -remove-tags generic -w

Add a location to a zone (and write the output):

zonedb -zones alsace -add-locations fr-a -w

Add zh-Hans-HK (Hong Kong Simplified Chinese) as a language (and write the output):

zonedb -zones 香港 -add-languages zh-Hans-HK -w

List all zones with only one language:

zonedb -grep '"languages"\: \[\s+\S+\s+\]'

List all zones with the geo tag that support a Chinese language variant:

zonedb  -tags geo -grep zh-

List all IDN zones with the geo tag:

zonedb -grep '[^\x{0000}-\x{007F}]' -tags geo

Or, using the -idn tag, all IDN TLDs with the geo tag:

zonedb -tlds -idn -tags geo

License

The ZoneDB database is licensed under the Open Database License (ODbL) version 1.0. See LICENSE-DATA.md for more information.

The Go package derived from this database is licensed under the MIT license. See LICENSE for more information.

Copyright © the Public Zone Database authors.