PostgresOSM.import_osm_data

PostgresOSM.import_osm_data(osm_data, table_name, schema_names=None, table_named_as_subregion=False, schema_named_as_layer=False, if_exists='fail', force_replace=False, chunk_size=None, confirmation_required=True, verbose=False, raise_error=False, **kwargs)

Import OSM data into a database.

Parameters:

• osm_data (dict) – OSM data of a geographic (sub)region

• table_name (str) – name of a table

• schema_names (list | dict | None) – names of schemas for each layer of the PBF data. Defaults to None; when schema_names=None, the default layer names as schema names

• table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to False

• schema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to False

• if_exists (str) – if the table already exists. Defaults to 'fail'; valid options include {'replace', 'append', 'fail'}

• force_replace (bool) – whether to force to replace existing table. Defaults to False

• chunk_size (int | None) – the number of rows in each batch to be written at a time, defaults to None

• confirmation_required (bool) – whether to prompt a message for confirmation to proceed, defaults to True

• verbose (bool) – whether to print relevant information in console as the function runs, defaults to False

• raise_error (bool) – Whether to raise the provided exception. If raise_error=False (default), the error will not be raised.

• kwargs – [optional] parameters of the method import_osm_layer()

Examples:

>>> from pydriosm.ios._base import BaseIOS
>>> from pyhelpers.dirs import delete_dir

>>> osmdb = BaseIOS(database_name='osmdb_test', verbose=True)
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

>>> subrgn_name = 'Rutland'  # name of a subregion
>>> dat_dir = "tests/osm_data"  # name of a data directory where the subregion data is

Example 1 - Import data of a PBF file:

>>> # First, read the PBF data of Rutland
>>> # (If the data file is not available, it'll be downloaded by confirmation)
>>> raw_rutland_pbf = osmdb.reader.read_pbf(
...     subrgn_name, dat_dir, download=True, verbose=True)
Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.92M/1.92M | 4.88MB/s ...
  Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done.
Reading "./tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done.
>>> type(raw_rutland_pbf)
dict
>>> list(raw_rutland_pbf.keys())
['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']

>>> # Import all layers of the raw PBF data of Rutland
>>> osmdb.import_osm_data(raw_rutland_pbf, table_name=subrgn_name, verbose=True)
Proceed to import data into table "Rutland" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
  "points" ... Done. (6473 features)
  "lines" ... Done. (11057 features)
  "multilinestrings" ... Done. (71 features)
  "multipolygons" ... Done. (9423 features)
  "other_relations" ... Done. (33 features)

>>> # Get parsed PBF data
>>> parsed_rutland_pbf = osmdb.reader.read_pbf(
...     subregion_name=subrgn_name, data_dir=dat_dir, expand=True,
...     parse_geometry=True, parse_other_tags=True, verbose=True)
Parsing "./tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done.
>>> type(parsed_rutland_pbf)
dict
>>> list(parsed_rutland_pbf.keys())
['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']

>>> # Import data of selected layers into specific schemas
>>> schemas = {
...     "schema_0": 'lines',
...     "schema_1": 'points',
...     "schema_2": 'multipolygons',
... }
>>> osmdb.import_osm_data(parsed_rutland_pbf, subrgn_name, schemas, verbose=True)
Proceed to import data into table "Rutland" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
  "schema_0" ... Done. (11057 features)
  "schema_1" ... Done. (6473 features)
  "schema_2" ... Done. (9423 features)

>>> # To drop the schemas "schema_0", "schema_1" and "schema_2"
>>> osmdb.drop_schema(schemas.keys(), confirmation_required=False, verbose=True)
Dropping the following schemas from postgres:***@localhost:5432/osmdb_test:
  "schema_0" ... Done.
  "schema_1" ... Done.
  "schema_2" ... Done.

Example 2 - Import OSM GeoPackage data:

>>> # Read GeoPackage data of Rutland
>>> rutland_gpkg = osmdb.reader.read_gpkg(
...     subregion_name=subrgn_name, data_dir=dat_dir, download=True, verbose=True)
Downloading "rutland-latest-free.gpkg.zip" 100%|██████████| 3.30M/3.30M | 5.0...
  Saving "rutland-latest-free.gpkg.zip" to "./tests/osm_data/rutland/" ... Done.
Parsing the data ... Done.
>>> type(rutland_gpkg)
dict
>>> list(rutland_gpkg.keys())
['traffic',
 'places',
 'pois',
 'transport',
 'pofw',
 'natural',
 'railways',
 'roads',
 'waterways',
 'protected_areas',
 'water',
 'landuse',
 'buildings',
 'adminareas']

>>> # Import all layers of the shapefile data of Rutland
>>> osmdb.import_osm_data(osm_data=rutland_gpkg, table_name=subrgn_name, verbose=True)
Proceed to import data into table "Rutland" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
  "traffic" ... Done. (557 features)
  "places" ... Done. (301 features)
  "pois" ... Done. (1081 features)
  "transport" ... Done. (65 features)
  "pofw" ... Done. (65 features)
  "natural" ... Done. (665 features)
  "railways" ... Done. (141 features)
  "roads" ... Done. (7315 features)
  "waterways" ... Done. (379 features)
  "protected_areas" ... Done. (20 features)
  "water" ... Done. (233 features)
  "landuse" ... Done. (2461 features)
  "buildings" ... Done. (5706 features)
  "adminareas" ... Done. (58 features)

Example 3 - Import BBBike shapefile data file of Leeds:

>>> # Change the data source
>>> osmdb.data_source = 'BBBike'
>>> subrgn_name = 'Leeds'

>>> # Read shapefile data of Leeds
>>> leeds_shp = osmdb.reader.read_shp(
...     subregion_name=subrgn_name, data_dir=dat_dir, download=True, rm_extracts=True,
...     verbose=True)
Downloading "Leeds.osm.shp.zip" 100%|██████████| 57.7M/57.7M | 20.6MB/s | ETA...
  Saving "Leeds.osm.shp.zip" to "./tests/osm_data/leeds/" ... Done.
Extracting "./tests/osm_data/leeds/Leeds.osm.shp.zip"
  to "./tests/osm_data/leeds/" ... Done.
Reading the shapefile(s) at "./tests/osm_data/leeds/Leeds-shp/shape/" ... Done.
Deleting the extracts "./tests/osm_data/leeds/Leeds-shp/" ... Done.
>>> type(leeds_shp)
dict
>>> list(leeds_shp.keys())
['buildings',
 'landuse',
 'natural',
 'places',
 'points',
 'railways',
 'roads',
 'waterways']

>>> # Import all layers of the shapefile data of Leeds
>>> osmdb.import_osm_data(osm_data=leeds_shp, table_name=subrgn_name, verbose=True)
Proceed to import data into table "Leeds" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
  "buildings" ... Done. (432701 features)
  "landuse" ... Done. (22180 features)
  "natural" ... Done. (7759 features)
  "places" ... Done. (892 features)
  "points" ... Done. (47332 features)
  "railways" ... Done. (2897 features)
  "roads" ... Done. (152155 features)
  "waterways" ... Done. (3520 features)

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
Drop the database "osmdb_test" from postgres:***@localhost:5432?
 [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(dat_dir, verbose=True)
To delete the directory "./tests/osm_data/" (Not empty)
? [No]|Yes: yes
Deleting "./tests/osm_data/" ... Done.