InternetData Documentation

Every database published today needs an API key carrying the db.download scope. Access is granted by contract, one family at a time, so there is no self-serve tier: write to dev@internetdata.io to be licensed and issued a key. api_key: is nevertheless optional - a client built without one sends no Authorization header at all, ready for a database served without a license.

require 'internetdata'

client = InternetData::Client.new(api_key: ENV['INTERNETDATA_API_KEY'])

client.database.list.each do |database|
  puts [database.base, database.standing, database.versions.map(&:id).join(', ')].join(' ')
end

list returns one entry per database FAMILY, because a license is held against the family while a download names a specific version. standing is licensed if the family is yours today, expired if the term has ended, and unlicensed if it is published but has never been bought. versions carries the ids you pass everywhere else, oldest first, and the formats each one is actually built in.

What is inside a database

meta = client.database.metadata('bogon_ip_v1')

meta.updated              # => '2026-09-04', the day this build was generated
meta.entries              # => rows in the build
meta.size['csvgz']        # => bytes you are about to move
meta.schema['csvgz']      # => [#<DatabaseMetadataColumn name="ip" type="string" ...>, ...]
meta.sample['mmdb']       # => a few real rows

Poll updated and entries to decide whether today's build is worth fetching; both are free of the transfer. size is also the honest way to budget a download before starting one.

Downloading

download streams a file to a path and returns the number of bytes written. Nothing larger than a chunk is ever held in memory, whatever the database weighs:

written = client.database.download('bogon_ip_v1', 'mmdb', './bogon_ip_v1.mmdb')

The bytes go to a neighboring .part file and the name only appears once the transfer completes, so an interruption cannot leave a truncated file that reads as a whole database, and a refresh that fails does not destroy the copy already there.

Or take the link and run the transfer yourself. The API answers a redirect to time-limited object storage, and that URL authorizes itself, so it can be handed to something holding no API key:

url = client.database.download_url('bogon_ip_v1', 'mmdb')

Or, for a small database, take the bytes directly:

bytes = client.database.download_bytes('bogon_asn_v1', 'csvgz')

download_bytes holds the whole file in memory and the catalog spans seven orders of magnitude, from a few hundred bytes to over 5 GiB, so use download for anything you have not checked with metadata first.

Verifying what you got

sums = client.database.checksums('bogon_ip_v1', 'mmdb')
sums.sha256   # also md5, sha1, sha512

Download history

Your organization's recent attempts, newest first, refusals included, because a denial is what answers "it stopped working" and its absence answers nothing:

client.database.downloads(limit: 20).each do |attempt|
  puts [attempt.created, attempt.dataset_id, attempt.format, attempt.outcome].join(' ')
end

Timeouts

client = InternetData::Client.new(api_key: ENV['INTERNETDATA_API_KEY'], timeout: 10)

catalog = client.database.list(timeout: 5)

timeout is in seconds and bounds each attempt, body included, so a call that is retried can take longer in total. It defaults to 30 seconds. The client's value is the default: from 2.1.0, list, metadata, checksums, downloads and download_url each take timeout: for that call alone.

download and download_bytes take no timeout: and raise ArgumentError if handed one, rather than accepting it and quietly doing nothing: a transfer runs to gigabytes and minutes, so any bound that suits a JSON call would abandon a healthy download. They bound only their connection with the client's value. download_url does take one, because minting the link is an ordinary API request - it bounds that request, not whatever you do with the link afterwards.

Errors

Failures raise an InternetData::Error carrying a kind and a retryable? flag:

begin
  client.database.download('vpn_ip_v1', 'mmdb', './vpn_ip_v1.mmdb')
rescue InternetData::Error => e
  warn "#{e.kind} #{e.status} #{e.retryable?}: #{e.rc}"
end

kind is one of :bad_request, :unauthorized, :forbidden, :rate_limited, :quota_exceeded, :server_error or :network. rc is the API's own result code, such as NOT_LICENSED or LICENSE_EXPIRED, which is usually the specific thing you want to read.

Note that :rate_limited and :quota_exceeded both arrive as HTTP 429 and are not the same thing. A rate limit is the API facing a burst, so retrying later works; a spent quota needs your allowance raised or the window to roll over. The library retries the first for you and never the second.

A program running on a person's own machine can let them sign in with their browser and pick one of their API keys, instead of asking them to paste one.

client = InternetData::Client.new
device = client.oauth.device_authorization('your-client-id', scope: 'account.read apikeys.read apikeys.reveal')
puts "Open #{device.verification_uri} and enter #{device.user_code}"

token = client.oauth.poll_device_token('your-client-id', device)
raise 'no API key was picked' if token.apikey.nil?

keyed = InternetData::Client.new(api_key: token.apikey)

poll_device_token raises InternetData::OauthAccessDeniedError when the person refuses and InternetData::OauthExpiredTokenError when the code expires first. Client IDs are issued on request from support@internetdata.io, and client.oauth.revoke('your-client-id', token.refresh_token) signs the machine out.

Ruby

Getting Started

Usage