jsonとyaml

jsonを使う機会が増えました。

jsonファイルから情報を取得するときに、jqコマンドが便利ですが、yamlのほうが見た目がわかりやすいのです。

jqコマンドと同じように操作できて、yamlとjsonに相互変換できるyqコマンドが便利です。

インストール

下記のコマンドでインストールすることができます。

pip install yq

jsonからyamlへの変換

オープンデータからjsonデータを取ってきます。

$ curl -s https://catalog.data.gov/harvest/object/345de16e-2fd5-4c52-9487-0467de11ad99 
{"accessLevel": "public", "bureauCode": ["005:12"], "contactPoint": {"@type": "vcard:Contact", "fn": "Brian Brotsos", "hasEmail": "mailto:brian.brotsos@usda.gov"}, "description": "USDA CIO Governance Board Membership List include all governance boards the CIO is a member of. Agencies shall keep this list up to date at least annually beginning in April 2016. This file is distributed for FITARA guidance https://management.cio.gov/ and more information on this format can be found at https://management.cio.gov/schema/", "distribution": [{"@type": "dcat:Distribution", "downloadURL": "http://www.usda.gov/digitalstrategy/governanceboards.json", "mediaType": "application/json", "title": "Governance Boards"}], "identifier": "usda-ocio-15-0008", "keyword": ["FITARA"], "landingPage": "http://www.usda.gov/wps/portal/usda/usdahome?navid=it-governance-boards", "license": "http://creativecommons.org/publicdomain/zero/1.0/", "modified": "2015-08-27", "programCode": ["005:059"], "publisher": {"@type": "org:Organization", "name": "Office of Chief Information Officer", "subOrganizationOf": {"@type": "org:Organization", "name": "Department of Agriculture"}}, "title": "USDA Governance Boards"}

これをyamlに整形します。

$ curl -s https://catalog.data.gov/harvest/object/345de16e-2fd5-4c52-9487-0467de11ad99 | yq -y '.'
accessLevel: public
bureauCode:
- 005:12
contactPoint:
  '@type': vcard:Contact
  fn: Brian Brotsos
  hasEmail: mailto:brian.brotsos@usda.gov
description: USDA CIO Governance Board Membership List include all governance boards
  the CIO is a member of. Agencies shall keep this list up to date at least annually
  beginning in April 2016. This file is distributed for FITARA guidance https://management.cio.gov/
  and more information on this format can be found at https://management.cio.gov/schema/
distribution:
- '@type': dcat:Distribution
  downloadURL: http://www.usda.gov/digitalstrategy/governanceboards.json
  mediaType: application/json
  title: Governance Boards
identifier: usda-ocio-15-0008
keyword:
- FITARA
landingPage: http://www.usda.gov/wps/portal/usda/usdahome?navid=it-governance-boards
license: http://creativecommons.org/publicdomain/zero/1.0/
modified: '2015-08-27'
programCode:
- 005:059
publisher:
  '@type': org:Organization
  name: Office of Chief Information Officer
  subOrganizationOf:
    '@type': org:Organization
    name: Department of Agriculture
title: USDA Governance Boards

yamlからjsonへの変換

yamlに変換したデータを再度、jsonに変換します。

jqコマンドのフィルタも、そのまま使えるのがいいですね。

$ curl -s https://catalog.data.gov/harvest/object/345de16e-2fd5-4c52-9487-0467de11ad99 | yq -y '.' | yq '.'
{
  "accessLevel": "public",
  "bureauCode": [
    "005:12"
  ],
  "contactPoint": {
    "@type": "vcard:Contact",
    "fn": "Brian Brotsos",
    "hasEmail": "mailto:brian.brotsos@usda.gov"
  },
  "description": "USDA CIO Governance Board Membership List include all governance boards the CIO is a member of. Agencies shall keep this list up to date at least annually beginning in April 2016. This file is distributed for FITARA guidance https://management.cio.gov/ and more information on this format can be found at https://management.cio.gov/schema/",
  "distribution": [
    {
      "@type": "dcat:Distribution",
      "downloadURL": "http://www.usda.gov/digitalstrategy/governanceboards.json",
      "mediaType": "application/json",
      "title": "Governance Boards"
    }
  ],
  "identifier": "usda-ocio-15-0008",
  "keyword": [
    "FITARA"
  ],
  "landingPage": "http://www.usda.gov/wps/portal/usda/usdahome?navid=it-governance-boards",
  "license": "http://creativecommons.org/publicdomain/zero/1.0/",
  "modified": "2015-08-27",
  "programCode": [
    "005:059"
  ],
  "publisher": {
    "@type": "org:Organization",
    "name": "Office of Chief Information Officer",
    "subOrganizationOf": {
      "@type": "org:Organization",
      "name": "Department of Agriculture"
    }
  },
  "title": "USDA Governance Boards"
}