Rust

Getting Started

cargo add internetdata

Requires Rust 1.85 or newer. Everything is async and runs on tokio.

Usage

Every database published today needs an API key carrying the db.download scope. Create one in the console, then pass it in. api_key is optional on the builder - leave it out and no Authorization header is sent at all, ready for a database served without a license:

use internetdata::Client;

let client = Client::builder()
    .api_key(std::env::var("INTERNETDATA_API_KEY")?)
    .build()?;

for database in client.database().list().await? {
    println!("{} ({}): {}", database.name, database.standing, database.summary);
}

The database calls hang off client.database(), which is where the sibling VPNDetection crate keeps the same seven.

Every setting has a default, and Client::builder() is where you change one:

let client = Client::builder().api_key(key).retries(4).build()?;

Each attempt at a request gives up after 30 seconds, and timeout changes that. A database download isn't bound by it, since a large one runs for minutes:

let client = Client::builder().api_key(key).timeout(Duration::from_secs(5)).build()?;

The catalog

list() answers database FAMILIES. A license covers the family, while a download names one of its versions, so the ID you pass to download, checksums and metadata comes from versions:

for database in client.database().list().await? {
    println!("{} is {}", database.base, database.standing);   // "bogon_ip is licensed"

    for version in &database.versions {
        println!("  {} built in {:?}", version.id, version.formats);
    }
}

standing is Licensed for a live grant, Expired for one whose term has ended, and Unlicensed for a database published but never bought, so you can see what else exists without asking us. license_type says what your license lets you do with the data, and is None when there is no license.

What is inside a database

metadata carries the column schema, a few real rows, the row count and the size of each artifact, without downloading anything. Poll it to decide whether today's build is worth fetching, and read size to know what a transfer will cost before you start it:

let meta = client.database().metadata("bogon_ip_v1").await?;

println!("{} rows, built {}", meta.entries, meta.updated);   // 1234 rows, built 2026-09-04
println!("{:?} bytes", meta.size.get("csvgz"));              // Some(760)

for column in &meta.schema["csvgz"] {
    println!("{}: {}", column.name, column.r#type);
}

Downloading

use internetdata::DatabaseFormat;

let written = client.database().download("bogon_ip_v1", DatabaseFormat::Csvgz, "./bogon_ip.csv.gz").await?;

download streams to disk through a neighboring .part file, so nothing bigger than a chunk is ever held in memory and a transfer that dies half way neither leaves a truncated file nor replaces the copy already on disk.

let raw = client.database().download_bytes("bogon_asn_v1", DatabaseFormat::Csvgz).await?;

download_bytes holds the whole file in memory. The catalog spans seven orders of magnitude, from a few hundred bytes to several gigabytes, so check metadata first for anything you have not measured.

let url = client.database().download_url("bogon_ip_v1", DatabaseFormat::Csvgz).await?;

download_url hands back the time-limited link the API redirects to, so you can run the transfer yourself with whatever tool you like. It authorizes itself and carries none of your credentials, so it is safe to pass on; the link authorizes the START of a transfer, so one already running is not interrupted when it lapses.

Each of the three downloads the build that is live unless you name one. A database.updated webhook names the build it announces and carries its sha256, so a receiver that downloads after the next publish asks for that build, through the _with form of the call, to get the file the hash is of:

use internetdata::DownloadOptions;

let options = DownloadOptions::new().build(build_from_the_webhook);
let written = client.database().download_with("vpn_ip_v1", DatabaseFormat::Mmdb, "./vpn_ip_v1.mmdb", &options).await?;

A build is kept 7 days. Past that the call fails as a BadRequest with the message BUILD_GONE, and downloading without one fetches the live build.

Not every database is built in every format, which is why versions lists the ones it has. Asking for another is a BadRequest, not a gap.

Verifying a download

let sums = client.database().checksums("bogon_ip_v1", DatabaseFormat::Csvgz).await?;
println!("{}", sums.sha256);

Download history

downloads lists your organization's recent attempts, newest first, refusals included. A denial is what answers "it stopped working", and its absence answers nothing:

for attempt in client.database().downloads(Some(20)).await? {
    println!("{} {} {:?} {}", attempt.created, attempt.dataset_id, attempt.outcome, attempt.http_status.unwrap_or(0));
}

Pass None for the API's own default of 50. Anything above 200 is clamped.

Errors

Failures return an internetdata::Error carrying a kind() and a retryable() flag:

use internetdata::ErrorKind;

match client.database().metadata("bogon_ip_v1").await {
    Ok(meta) => println!("{}", meta.entries),
    Err(err) => println!("{} {}", err.kind(), err.retryable()),
}

kind() is one of BadRequest, Unauthorized, Forbidden, RateLimited, QuotaExceeded, ServerError, Network or Io, the last of which is a database transfer that could not be written or that ended early. message() carries the API's own reason, such as NOT_LICENSED or UNKNOWN_DATASET.

RateLimited is a 429 that carries Retry-After, and the library retries it for you. QuotaExceeded is a 429 without one, which it never retries. InternetData's database API has no quota, so you won't see QuotaExceeded from it.

TLS backends

rustls is the default, so the crate builds with no system libraries at all. If you would rather link the platform's TLS, or you already depend on reqwest with its own defaults and want one backend rather than two:

internetdata = { version = "2", default-features = false, features = ["native-tls"] }

Calling from synchronous code

There is no blocking facade, on purpose: reqwest::blocking builds its own runtime and panics when constructed inside one, so a facade would fail for exactly the callers most likely to reach for it. If you have no runtime, make the cost visible instead:

let runtime = tokio::runtime::Runtime::new()?;
let client = Client::builder().api_key(key).build()?;
let databases = runtime.block_on(client.database().list())?;

Sign in with OAuth (device flow)

A program running on the person's own machine can let them sign in with a browser and pick one of their API keys, instead of asking them to paste it:

use internetdata::{Client, DeviceAuthorizationOptions};

let client = Client::new()?;
let scope = DeviceAuthorizationOptions::new().scope("account.read apikeys.read apikeys.reveal");
let device = client.oauth().device_authorization_with("your-client-id", scope).await?;
println!("Open {} and enter {}", device.verification_uri, device.user_code);

let token = client.oauth().poll_device_token("your-client-id", &device).await?;
let Some(apikey) = token.apikey else {
    return Err("no API key came back: none was picked, or it cannot be shown again".into());
};
let keyed = Client::builder().api_key(apikey).build()?;

A denied sign-in fails with OauthError::AccessDenied and a code that ran out with OauthError::ExpiredToken. Client IDs are issued on request from support@internetdata.io, and client.oauth().revoke("your-client-id", &refresh_token) signs the machine out again.

Sign in with OAuth (authorization code)

An app that can take a browser redirect signs the person in there instead, with a PKCE pair made for that one sign-in:

use internetdata::{AuthorizationUrlOptions, Client};

let client = Client::new()?;
let redirect_uri = "http://127.0.0.1:8765/callback";
let pkce = client.oauth().create_pkce()?;

let opts = AuthorizationUrlOptions::new().scope("apikeys.use").state("your-state");
let url =
    client.oauth().authorization_url_with("your-client-id", redirect_uri, &pkce.challenge, opts)?;
// Open url in the browser. Its redirect to redirect_uri carries code and state.
let token = client
    .oauth()
    .exchange_authorization_code("your-client-id", &code, &pkce.verifier, redirect_uri)
    .await?;

Check that state came back as you sent it before you exchange code, which works once. The client ID can also be the https URL of a client metadata document your app serves, and such an app is never handed a key, so token.apikey stays None.

Register a client

An app with no client ID, and no client metadata document to use as one, registers itself, then signs in with the client_id it gets back as the client ID for authorization_url and exchange_authorization_code:

let client = Client::new()?;
let opts = RegisterOptions::new().client_name("Your App");
let registration =
    client.oauth().register_with(&["http://127.0.0.1:8765/callback"], opts).await?;

Each redirect URI must be https on a public host name, or http on a loopback address, which then matches any port. Nothing is kept until a person approves the client, so an app with no client ID saved can simply register again.