Welcome to the Open Data Formats initiative, an effort to promote the use of license free, public standards for the encoding and storage of data.
By 2025, 463 exabytes of data will be created every day and much of it will be stored in immutable storage systems such as a blockchain. As systems move toward a more microservice based architecture, data in transit, between disparate providers, will also be loosely defined.
There is a need for data providers to explicitly state the format of their data, how it is encoded, for consumers, present and future. Data stored long term, such as on a blockchain, could outlive the original creator; the project is no longer maintained, the company goes out of business, etc. It will be important for future consumers of the data to know how to properly decode it.
Open Data Formats aims to provide a repository for licence free, open source, structured data formats.
An “Interface Definition Language” defines the structure, encoding, and decoding method for data into programming language, implementation specific formats.
A “Tier 1 IDL” is one that is mature and compiles to a majority of the top programming languages in use. The following are the tier 1 interface definition languages Open Data Formats promotes.
ECMAScript/JS 6 Node library for creating compact, portable Open Timestamps attestations.
You can create and decode SimpleStamp files using the online tool.
24,343 world cities and 252 countries.
A convenient wrapper for the GeoNames database of world cities data.
The data from the GeoNames tab seperated values files are parsed and normalized into a JSON object for quick lookup and distance calculations. The data for each city's country is also normalized, indexed by ISO_3166-1 country code, paired with the country's flag's SVG, GeoJSON of its boundaries, and relative URL to Wikipedia page.
Open data formats can be used by any project, open source or totally private. The only license for submitted formats is the most open and non-restrictive, dependant projects cannot be encumbered by a license on the data itself.
A project that is completely closed and never intends to share data still has incentive to use open data formats, rather than redefine common data structures. Any project that is a collaboration between another, such as two companies developing a shared API, can use open data formats as well, to serve as a “promise” to one another that the data itself will be accessible, regardless of circumstances.
Proposed Licenses:
There are existing data formats that are open and license free on Wikipedia at List of open formats.
The Open Knowledge Foundation maintains a handbook and guide for opening up data access in general, including how to and advocacy.
Open Data Formats is a work in progress. Contact us for more information
info@opendataformats.org