Censys Search

Rumble supports import of data from Censys Search. There are two ways to obtain data:

  • API integration
  • Data import from Censys Avro files.

Getting started

To get started with the Censys Search API, you will need to register for a Censys Search account. Once you have done so, you can find your API credentials in the My Account section.

In Rumble, go to the Credentials page, and click Add Credential. Select Censys Search API Key as the credential type, and enter your API ID and API secret.

You can now go to your asset inventory, click the Connect button, and choose Censys Search API. Select the credential you just created from the Censys Search credential dropdown.

Censys search configuration

There are two modes for connecting Rumble to the Censys Search API.

  • Custom Query mode - Rumble runs a Censys search query you specify, and then imports all of the results into Rumble. The search query should be in Censys Search Language. It is a good idea to test your query using the main Censys Search 2.0 interface before running an import task.

  • All Assets mode - Rumble assembles a list of public IP addresses from all of the assets in the selected site, and then uses the API to find Censys Search information for those addresses. The information found is imported into Rumble and merged into the appropriate assets.

As with a Rumble scan, you’ll need to select a site to contain the scan data. The usual task scheduling options are available.

When you have finished editing the Censys Search configuration, click Activate Connection.

Censys Avro data import

As an alternative to use of the Censys Search API, Rumble can import data from Censys data files. These are obtained directly from Censys as part of a subscription service, and are in Apache Avro format.

To process Censys data files, you use the Rumble scanner’s censys command. It takes any number of arguments, which can be:

  • Names of Avro files, which must end in .avro
  • CIDRs or IP addresses to search for in the files

The scanner reads the Avro files specified, and writes a file in Rumble scan format containing the appropriate host records. By default, the file has a name matching censys-*.rumble.gz and is written to the current directory. Alternatively you can specify an output filename with the --output-raw option, as if performing a Rumble scan.

The Rumble scan file can be uploaded to the Rumble console like any other scan file.

If you have more IP addresses or CIDRs than will fit on a command linem, you can use the --input-targets option to specify that the scanner should read them from a file. The file is expected to be ASCII text, and contain CIDRs or IP addresses separated by whitespace (which can include newlines).

You can also use the scanner to process data, upload it, and then delete the scan data file if everything succeeded. For example:

% ./rumble-scanner censys universal-internet-dataset-20210923-000000000000.avro \ --upload --api-key=YOUR_ORGANIZATION_API_KEY \
  --upload-site="Primary site"

If you are using self-hosted Rumble, you can use the --api-url option to specify your console’s API endpoint.

The censys command also supports the --verbose option, which will make it list host addresses as they are written to the output file.