Skip to content

Latest commit

 

History

History
283 lines (198 loc) · 11.3 KB

metadata.md

File metadata and controls

283 lines (198 loc) · 11.3 KB

Metadata


HDX Metadata

All of the data in HDX HAPI comes from publicly-available datasets on HDX, and the HDX metadata can be used refer back to the original data to see what has been simplified or omitted in the API. The two primary HDX metadata tables are dataset and resource, and they contain a small subset of the metadata available through the CKAN API or HDX library.

The HAPI data records contain enough information to access the full records from CKAN (see the parameters tables for more details). All subcategory tables link back to a resource record, and all resources link back to a dataset record.

Dataset

Used in: Resource, all sub-category tables

This table contains the HDX-specific metadata associated with all datasets used to populate the HAPI sub-category tables. Every dataset has at least one child resource.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/dataset_parameters.yaml') }}

Resource

Used in: All sub-category tables

This table contains the HDX-specific metadata associated with all resources used to populate the HAPI sub-category tables. Every resource has one parent dataset.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/resource_parameters.yaml') }}

Geographical Metadata

HAPI supports three hierarchical levels of geographical metadata: location (a country or country-like entity), admin 1, and admin 2. An entry in the location table does not necessarily imply UN recognition of statehood. The subcategory data tables link to the lowest administrative level used by that data type; it will usually be admin 2, but in some cases may be admin 1 or location.

The names and p-codes are read in from a global p-code list taken from the common operational dataset (COD) gazetteers and administrative boundaries. In the geographical tables, the code (p-code) field is unique only in combination with reference_period_start, since p-codes may be reused in different versions of geographical metadata.

When bringing in data from sub-categories, p-codes are used if present. Sometimes, these p-codes may not strictly follow the standard with variations such as shorter or longer country or admin unit codes. HDX HAPI utilises p-code handling provided by the hdx-python-country library, which has a p-code length matching algorithm to ensure correct admin units are determined.

This algorithm works as follows: First, a p-code is verified against the expected p-code format. Then, a length-matching check is performed, ensuring that for each segment (country code or admin unit code), the length matches the predefined format for the country. For example, a 3-letter country code could be converted to a 2-letter one or vice-versa. Digits can be prefixed by 0's to increase the length, or have 0's dropped from the prefix.

Location

Country or country-like entities in HDX HAPI, from the CODs global p-code list.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/location_parameters.yaml') }}

Admin 1

Admin 1 level names and p-codes in HDX HAPI, from the CODs global p-code list.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/admin1_parameters.yaml') }}

Admin 2

Admin 2 level names and p-codes in HDX HAPI, from the CODs global p-code list.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/admin2_parameters.yaml') }}

Sub-category Metadata

Currency

Used in: Food Prices

The currency table is populated using the WFP VAM Data Bridges API.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/currency_parameters.yaml') }}

Org

Used in: Who is Doing What Where - Operational Presence

The organization table is populated from the 3W data, using the following methodology:

  • Organization name and acronym strings are normalised. If an acronym isn’t available, the first 32 characters of the name are used.
  • This organization mapping is used for common alternative names
  • Organizations must have an associated organization type. If available, the organization type code is taken directly from the 3W data, otherwise the name string is normalised and matched to the org type names. In the absence of a direct match, phonetic matching is used for strings > 5 characters. If no match is found, the organization is skipped.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/org_parameters.yaml') }}

Org Type

Used in: Org, Who is Doing What Where - Operational Presence

The table is initially populated using the OCHA Digital Services organization types list. The following rows are then added:

Code Description
501 Civil Society
502 Observer
503 Development Programme
504 Local NGO

Organization types all have an associated description and code.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/org_type_parameters.yaml') }}

Sector

Used in: Who is Doing What Where - Operational Presence, Humanitarian Needs

This table is initially populated using the Global Coordination Groups dataset. The following rows are then added:

Code Name
Cash Cash programming
Hum Humanitarian assistance (unspecified)
Multi Multi-sector (unspecified)
Intersectoral Intersectoral

Sectors all have an associated name and code.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/sector_parameters.yaml') }}

WFP Commodity

Used in: Food Prices

The commodity table tracks all food items, and their associated commodity category, present in the food prices data.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/wfp_commodity_parameters.yaml') }}

WFP Market

Used in: Food Prices

Markets are defined as the physical locations where buyers and sellers come together to trade goods and services. This table is populated from the food prices data, which contains location names down to admin 2 as well as a latitude and longitude, but is not p-coded. Consequently, admin 1 and 2 names must be matched to p-codes using the algorithm provided by the hdx-python-country library which uses phonetic name matching and manual overrides.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox.

{{ read_yaml('data_usage_guides/endpoint_parameters/wfp_market_parameters.yaml') }}

HDX HAPI Metadata

Data Availability

The data availability endpoint in HDX HAPI shows what data is accessible by sub-category and administrative level. If relevant data exists for a specific sub-category and region, it will be included in the query results, along with the date that it was last updated in HDX HAPI.

Where data are available to the country (admin 0) level, then the admin 1 and 2 code and name fields will contain empty strings. Where the data are available to the admin 1 level, then the admin 2 code and name fields will contain empty strings. Note that some sub-categories can have data available at multiple administrative levels.

Parameters Returned

The table below describes the parameters returned from this endpoint. For available query parameters, please see the API sandbox

{{ read_yaml('data_usage_guides/endpoint_parameters/data_availability_parameters.yaml') }}