Namespace Utility Modules (NUM) Specification
Introduction
Millions of organisations and individuals have used their public facing website to publish useful data about themselves – e.g. contact information, logos, registration numbers and more. However, data presented on the web in this way is often not structured or easily machine readable. Web standards for structuring this data have a high technical barrier for adoption and can be inefficient when they require full HTML documents to be downloaded and parsed.
Many service providers collect structured and unstructured data from the web for indexing and inclusion in social graphs and knowledge graphs. However, the data isn't made freely available and each provider requires the data subject to use a different method to manage and update it.
The Namespace Utility Modules (NUM) protocol provides a way to store structured data in, and retrieve structured data from, the DNS. Data can be stored for any domain name or email address. Data is stored in DNS TXT records known as NUM records. Records are located using a NUM URI which is resolved using a series of DNS queries known as a NUM lookup.
Use cases for NUM records are standardised with modules. A NUM lookup for a custom record makes a query to the authoritative nameserver only. A lookup for a standardised record (a record based on a module) first queries the authoritative nameserver and if no record is returned, a query is then made to the NUM Server – a DNS-based store of NUM records that organisations and individuals can use to adopt the NUM protocol through a simple, user-friendly web interface.
Status
The NUM protocol specification is currently a draft, we are inviting members of the technical community to provide feedback – to do that please contact us .
Conventions in this Document
The key words "must", "must not", "required", "shall", "shall not", "should", "should not", "recommended", "may", and "optional" are to be interpreted as described in RFC2119 .
Links in this Document
For the benefit of readers, these icons are displayed next to links to indicate where the link leads:
Audience
This specification is written for readers looking for an in-depth understanding of the NUM protocol – how records are stored in the DNS and how they are retrieved by NUM clients. For a much lighter read, please see the technical introduction .
Author
NUM was created by Elliott Brown of NUM Technology Ltd.
Example Zone
To help understand how NUM fits into the DNS, we have illustrated an example zone:
Click the thumbnail to expand below or download this PDF .You can view the example zone in this PDF .
The zone shows how NUM fits into the DNS for the domain name numexample.com
, this domain name is used throughout this specification and many of the examples used in the specification are reflected in this example zone.
NUM ID
NUM uses unique identifiers to store and retrieve data known as NUM IDs. Domains and email addresses are valid NUM IDs. Web addresses are not valid NUM IDs but can be resolved as NUM URIs , using num://
instead of http://
or https://
.
NUM Domain
For the purposes of this specification, the domain name in the NUM ID is known as the NUM Domain. For example, the NUM IDs numexample.com
and alice@numexample.com
both share the same NUM Domain – numexample.com
.
NUM Module
Modules can be used to standardise particular use cases – this helps facilitate adoption on both sides: server and client. Modules can also make object description more efficient. Some examples are shown below.
NUM records are not limited to storing particular types of data, any data can be stored using a custom NUM record.
Example Modules
These modules demonstrate some ways that the NUM protocol can be used:
Contacts
Company contact information on the web is fragmented and unstructured. Storing this data in a NUM record makes it fast, efficient and machine-readable. NUM Contacts creates lots of new possibilities, like dialling a company domain name as if it were a telephone number and working through call centre options on your touch screen device before even starting your call.
Registrant
The growth of invalid, incorrect and proxy domain name registrant data has undermined the principles of the WHOIS protocol. More recently, GDPR has all but destroyed it. NUM Registrant records provide a way for registrants, not registrars, to store and serve registrant data – using the DNS.
Custodian
Proving ownership and/or control over a domain name is complex. Many service providers require business owners, website administrators and marketing teams to add records to their DNS to prove authority for a domain name. With NUM Custodian records, a domain administrator can store a list of hashed identifiers (e.g. email addresses) in their DNS once so that all service providers can easily verify who's a custodian for a domain name. This module is currently a proposal.
Creating a Module
Module configuration files are written in MODL . Once the module is published it can be adopted by anyone, either independently in their own DNS or through the user-friendly NUM Server service.
NUM URI
NUM data is located with URIs. NUM records can be structured in a similar way to webpages on the World Wide Web and retrieved using the same URI path concept. NUM URIs follow typical URI syntax but re-purpose the port
to indicate the module
number. NUM URIs take the following structure:
For example:
NUM allows email addresses to be resolved and also contain a path, e.g.: john.smith@numexample.com/path
.
When no module
is specified 0
is assumed. When no path
is specified /
is assumed. A module is often specified by a NUM client.
The subdomain www
and the trailing dot (.
) in a fully qualified domain name (FQDN) must be removed from NUM URIs by Clients. The NUM URI www.numexample.com
, www.numexample.com.
and numexample.com.
must be equivalent.
NUM Record
NUM records are supported by all existing DNS infrastructure and can be created on any DNS server. NUM records can be created using DNS management tools provided by all cloud DNS providers and most domain name registrars.
Resource Record Type
A NUM record is a standard DNS TXT resource record.
Record Format
NUM records must be written in MODL – a compact data serialisation language well suited to storing objects in the DNS. NUM records must specify a version number using the syntax @n=<version-number>
, the current version number is 1.
Due to DNS TXT record limitations, NUM records that exceed 255 characters in length must be split into multiple TXT record strings. To help overcome limits imposed by some DNS implementations, very large NUM records may be stored as record sets .
Record Locations
NUM records for a particular module can be stored in two parts of the DNS, known as NUM Zones: the Independent NUM Zone and the Hosted NUM Zone. Custom NUM records (records not based on a module) can only be stored in the Independent NUM Zone.
Independent NUM Zone
The Independent NUM Zone is found on the authoritative nameserver for the NUM Domain under the label _num
. The Independent NUM Zone for numexample.com
and alice@numexample.com
is _num.numexample.com
.
Hosted NUM Zone
All Hosted NUM Zones are stored in the DNS below the domain name num.net
. The Hosted NUM Zone for numexample.com
and alice@numexample.com
is _numexample.com.c.7.m.num.net
. The NUM Server automatically deals with publishing and managing NUM records using a zone distribution technique .
NUM records can be created and managed using a simple interface provided by NUM Server .
Email Section
NUM records for email addresses are stored in a separate section of the NUM Zone under the label e
. Mailbox names (e.g. alice
in alice@numexample.com
) are prefixed with an underscore. The DNS name of an email section of a NUM Zone is built like this:
For example: _alice.e._num.numexample.com
or _alice.e._numexample.com.c.7.m.num.net
.
NUM Zones for IDNs
NUM records for Internationalized Domain Names (IDNs) are stored in the same way as any other domain. IDN standards dictate that domain names containing non-ASCII characters are stored within the DNS in Punycode format and prepended with the string xn--
.
In the example num例.com
(例
is Chinese for example), the domain name is stored in the DNS as xn--num-xc0e.com
and so the Indepdendent NUM Zone is at _num.xn--num-xc0e.com
and the Hosted NUM Zone is at _xn--num-xc0e.com.n.f.5.num.net
.
Record Hierarchy
NUM imposes no limit on the number of records that can be stored in a NUM Zone. There are two types of record:
Root Records
A NUM record returned for a NUM URI with no path (e.g. num://numexample.com:1/
) is known as a root record. The root record for the Contacts module is known as the Contacts root record.
Branch Records
A NUM record returned for a NUM URI including a path (e.g. num://numexample.com:1/sales
) is known as a branch record. Modules use branch records in different ways – some rely heavily on navigation between records (e.g. Contacts), others not at all.
NUM Record Sets
Some domain name registrars and DNS service providers set maximum limits on DNS TXT records (usually over 1000 characters). To overcome this limit, NUM records can be split over multiple resource records at the same DNS location, by creating a DNS Resource Record Set. The first record must start with the syntax <record-number>/<record-total>|
, e.g. 1/3|
. Each subsequent record must start with the syntax <record-number>|
, e.g. 2|
. In this example, we've set an arbitrary limit of 27 characters to demonstrate:
NUM imposes no limit on the number of records in a set but some DNS server and client implementations may have difficulties processing very large record sets.
Reserved Keys
Keys prefixed with @
may have special meaning.
The following keys are reserved for NUM, they are case sensitive:
Key | Name | Description |
@n | NUM Version | This should be present in all NUM records, the value specifies the version of the NUM protocol. |
@L | Link | Included in a NUM record to indicate that one part of a NUM record is linked to another NUM record . |
@R | Redirect | Included in a NUM record to instruct the NUM library to redirect to another NUM lookup . |
Some modules may also use JSON-LD reserved keywords .
NUM Lookup
NUM URIs are resolved using a NUM lookup, which involves a series of DNS TXT record queries. NUM lookups are made by NUM clients through NUM libraries . The structure of a DNS query made as a result of a NUM Lookup is as follows:
This structure is explained by way of example in the following sections.
NUM Queries
A lookup for a custom record must only query the Independent NUM Zone. A lookup for a standardised record (one based on a module) must run queries to the Independent and Hosted NUM Zones.
Independent Record Query
A lookup for the URI num://numexample.com:1/
results in a query to the Independent NUM Zone:
- Command Prompt
- Terminal
All queries for records based on a module require a query to the Independent NUM Zone.
Hosted Record Query
If a valid NUM record is not returned from the Independent NUM Zone, a query must be made to the Hosted NUM Zone. Hosted records are always found below num.net
. The DNS location of a hosted record is determined based on a zone distribution technique to help spread resource record load.
A lookup for the URI num://numexample.com:1/
results in a query to the Hosted NUM Zone:
- Command Prompt
- Terminal
All modules require a query to the Hosted NUM Zone.
Email Lookups
NUM Lookups for email addresses make queries to the email section of the NUM Zone. A NUM Lookup for the URI num://alice@numexample.com:1/
first runs a query to the Independent NUM Zone:
- Command Prompt
- Terminal
If a valid NUM record is not returned, a DNS query is made to the Hosted NUM Zone:
- Command Prompt
- Terminal
If a zone distribution record is returned by the independent or hosted query, a new query must be made using zone distribution . If the hosted query fails, the lookup fails.
Branch Record Lookups
When a NUM URI includes a path, it requires a branch record lookup. The URI path must be converted to a DNS name as follows:
- Replace path separators (
/
) with dots - Reverse the order of each part
For example, the URI num://numexample.com:1/foo/bar
will result in the following DNS query to the Independent NUM Zone:
- Command Prompt
- Terminal
If a valid NUM record is not returned, a DNS query is made to the Hosted NUM Zone:
- Command Prompt
- Terminal
NUM Responses
NUM responses are returned by DNS servers to NUM clients. Queries to the Hosted NUM Zone will either return a NUM record or NXDOMAIN
. In the event a query to the Hosted NUM Zone returns SERVFAIL
, the query should be retried at least once.
Client Side Processing
A key principle of NUM is that minimal user data leaves the client. DNS queries derived from the NUM URI and the client IP are the only pieces of data that should be passed to the DNS resolver. Aside from anycast implementations, the same NUM records should be delivered to all users worldwide. These records are then interpreted client side and displayed to users depending on user variables, for example – a different Contacts record can be displayed to people of different countries or languages, this is achieved with redirects .
DNS Responses
DNS responses must be received within 1.5 seconds, otherwise should be considered a failure. DNS responses must have a return message of NOERROR
(return code of 0
) and must return resource record(s) of type TXT. Any DNS Server response other than NOERROR
or returning records other than TXT
must be considered a failure.
Record Sets
If the response contains a NUM record set (more than one resource record) – the response must be assembled. Any record not starting with the syntax specified must be ignored. If a record in the set is missing or multiple records are found with the same record number, this must be considered a failure. In the example below, we've set an arbitrary limit of 27 characters to demonstrate. Notice there's no guarantee of the order in which records are returned:
Multi-string TXT Records
Due to DNS TXT record limitations , the contents of records over 255 characters must be stored as multiple TXT strings containing no more than 255 characters. TXT records that contain multiple TXT strings must be concatenated by the NUM client.
Links and Redirects
Links and Redirects are built into the NUM protocol. Links allow objects inside NUM records to link to other NUM records – indicating relationships or aiding navigation. Redirects can be used to re-route users to particular records.
Linking Records
The key @L
in any unpacked object indicates a link between NUM records. These links may be interpreted differently depending on the module. In the Contacts module it indicates a relationship between entities, e.g. a link between an organisation and a department or employee. Clients should allow users to follow these links to discover more data and may automatically follow these linked records to present more data to the user.
Valid Link Values
The link value must be a NUM URI. For links to records for the same NUM ID and module, relative URIs may be used. For links to records for a different NUM ID or module an absolute URI must be used, the protocol (num://
) may be omitted.
Redirecting Records
The key @R
indicates that the NUM library must redirect the NUM lookup. Redirects are used to redirect a client from the record specified in the URI to another record, this must happen before the user is shown any data – they must not see any part of the original record but must be shown the new URI that they have been redirected to.
Clients must not allow recursive redirects – where the redirect points at itself. To prevent redirection loops clients must allow a maximum of three redirects. Note that the lookup / query must only be processed if the key is present in the unpacked object , this allows for conditional redirects of lookups and queries.
Valid Redirect Values
The redirect value must be a valid NUM URI. For links to records for the same NUM ID and module relative URIs may be used. For links to records for a different NUM ID or module an absolute URI must be used, the protocol (num://
) may be omitted.
Security Considerations
NUM is built on top of the DNS protocol, known issues with DNS related to security are addressed by technologies such as DoH , DoT and DNSSEC (RFC 4033, RFC 4034, and RFC 4035). NUM is compatible with all of these technologies. Known issues related to user privacy are addressed in DPRIVE . The method of DNS resolution and the provider of a DNS resolver is a user decision.
Very careful consideration will be given to any module that provides payment or security information (e.g. bank details / PGP keys), NUM Records using these modules could be a target for attack.
A user request for the URI num://example.com
results in two queries: the first to the DNS of the authoritative zone for the domain numexample.com
and the second to the NUM Server under the domain num.net
, in the case of numexample.com
, NUM records will be served from _numexample.com.c.7.m.num.net
.
The NUM Server is a DNS-based store of NUM records which can be published and updated using a user-friendly interface. To publish or update records for a given domain a user must prove rights to do so. These rights are established through industry standard techniques: DNS TXT record creation; webpage creation on the webserver; homepage HTML modification.
DNS Limitations
The DNS is ultra-fast and reliable because it's light weight and its data is distributed and cached globally. These strengths have enabled the DNS to scale gracefully with the exponential growth of the internet and are what make the DNS an ideal location to store structured data. However, these same strengths mean the DNS is not well suited to providing results local to a user or providing a large dataset.
The following sections explain how these limitations can be overcome.
TXT Record Limit
DNS TXT record strings can be no longer than 255 characters. However, a single DNS TXT record can be composed of more than one string, as per RFC1035 sections 3.3.14 and 3.3. All DNS implementations should support multi-string TXT records but domain administrators creating independent records should check with their domain registrar / DNS provider.
UDP Packet Limit
NUM records are DNS TXT records and are typically transported via the UDP protocol. The DNS UDP packet limit was originally 512 bytes but almost all DNS clients and resolvers now accept UDP packets over 512 bytes through EDNS . Much larger DNS packets can be sent and received via TCP, but this may not be supported by all DNS clients, resolvers and servers.
To ensure maximum efficiency and compatibility with all hardware and software, it is recommended that NUM record responses are limited to 512 bytes where possible. This includes the DNS packet header and fully qualified domain name. The NUM record limit for the domain numexample.com
is calculated as follows:
Due to the efficiencies of the MODL language , a huge amount of data can be contained within a 512 byte NUM record.
Local Results
As the DNS is cached globally and no user data leaves the client, NUM is not well suited to returning data based on a user's location. Some modules use conditional logic to display different records to users dependent on their country and this can work well but showing records dependent on distance (for example your closest store) is more challenging. For example, a large nationwide brand might like to use the Contacts
module to store contact data about each of their 5,000 stores. NUM is well suited to storing the data – requiring 5,000 branch records, but it is not well suited to searching that data or for example, ordering it based on distance to the user. Third party services would be required for this.
Time Sensitive Results
DNS caching means NUM is best suited to data which changes relatively rarely – perhaps no more than once a week and certainly no more than once a day. NUM is ideally suited to some use cases e.g. contact data; and terribly suited to others e.g. live stock prices.
Zone Distribution Technique
Most DNS server implementations limit the size of a zone file and many DNS cloud providers limit the number of resource records that can be stored in a zone – for example, AWS Route 53, Google Cloud DNS and Azure Cloud DNS all set default limits of 10,000 resource records per zone. The vast majority of businesses would never reach these limits, but the largest businesses might. To ensure NUM is massively scalable, fast and reliable we have developed a zone distribution technique to spread records across multiple zones where necessary. This technique is used by the NUM Server to serve out responses for millions of domain names. It can also be deployed independently when hosting NUM records for a huge number of email users on the same domain.
Hosted Zone
For hosted record queries, the domain (after punycoding for an IDN ) is hashed using the SHA-1 algorithm and then Base36 encoded, it's truncated to 3 characters and these three characters are split into labels prepending num.net
in reverse order.
Email Section
Zone distribution in the email section of a NUM Zone is optional and can be deployed at 1-3 levels, this is only required for NUM Domains with tens of thousands of email addresses.
Zone Distribution Record
To store many thousands of NUM records for email addresses on the same domain, a wildcard DNS record must be setup at the root of the email section (below e
), specifying the level of zone distribution being used. An example zone distribution record in zone file format:
The zd
key indicates the levels of zone distribution required for email queries. The value must be 1-3.
Email record distribution is used on the NUM Server for some domains, e.g. large email providers like Gmail. When activated, it must be combined with NUM Server record distribution.