PostgresOSM.fetch_data¶

PostgresOSM.fetch_data(subregion_name, layer_names=None, chunk_size=None, method='tempfile', max_size_spooled=1, decode_geojson=True, sort_by='id', table_named_as_subregion=False, schema_named_as_layer=False, verbose=False, raise_error=False, **kwargs)[source]¶

Fetch OSM data (of one or multiple layers) of a geographic (sub)region.

See also [ROP-1].

Parameters:

subregion_name (str) – Name of a geographic (sub)region (or the corresponding table).
layer_names (list | None) – Names of schemas for each layer of the PBF data, if None (default), the default layer names as schema names.
chunk_size (int | None) – The number of rows in each batch to be written at a time. Defaults to None.
method (str | None) – Method to be used for buffering temporary data. Defaults to 'tempfile'.
max_size_spooled (int | float) – see pyhelpers.sql.PostgreSQL.read_sql_query(), Defaults to 1 (in GB).
decode_geojson (bool) – Whether to decode string GeoJSON data. Defaults to True.
sort_by (str | list) – Column name(s) by which the data (fetched from PostgreSQL) is sorted. Defaults to 'id'.
table_named_as_subregion (bool) – Whether to use subregion name as a table name. dDefaults to False
schema_named_as_layer (bool) – whether a schema is named as a layer name. Defaults to False.
verbose (bool | int) – whether to print relevant information in console. Defaults to False.
raise_error (bool) – Whether to raise the provided exception; if raise_error=False (default), the error will be suppressed.

Returns:

PBF (.osm.pbf) data

Return type:

dict

Examples:

>>> from pydriosm.ios import PostgresOSM
>>> from pyhelpers.dirs import delete_dir

>>> osmdb = PostgresOSM(database_name='osmdb_test', verbose=True)
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

>>> subregion_name = 'Rutland'  # name of a subregion
>>> data_dir = "tests/osm_data"  # name of a data directory where subregion data is

>>> # Import PBF data of Rutland
>>> osmdb.import_osm_pbf(subregion_name, data_dir=data_dir, verbose=True)
Proceed to import .osm.pbf data of the following geographic (sub)region(s):
    "Rutland"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.94M/1.94M | 6.46MB/s ...
  Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done.
Reading "tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done.
Importing the data into the table "Rutland" ...
  "points" ... Done. (6578 features)
  "lines" ... Done. (11231 features)
  "multilinestrings" ... Done. (70 features)
  "multipolygons" ... Done. (9466 features)
  "other_relations" ... Done. (33 features)

>>> # Import shapefile data of Rutland
>>> rutland_shp = osmdb.reader.read_shp(
...     subregion_name=subregion_name,
...     data_dir=data_dir,
...     rm_extracts=True,
...     verbose=True
... )
>>> rutland_shp is None
True

>>> # Import geopackage data of Rutland
>>> rutland_gpkg = osmdb.reader.read_gpkg(
...     subregion_name=subregion_name,
...     data_dir=data_dir,
...     download=True,
...     verbose=True
... )
Downloading "rutland-latest-free.gpkg.zip" 100%|██████████| 3.32M/3.32M | 7.5...
  Saving "rutland-latest-free.gpkg.zip" to "./tests/osm_data/rutland/" ... Done.
Parsing the data ... Done.

>>> osmdb.import_osm_data(rutland_gpkg, table_name=subregion_name, verbose=True)
Proceed to import data into the table "Rutland" at postgres:***@localhost:5432/osmd...
 [No]|Yes: yes
Importing the data ...
  "traffic" ... Done. (589 features)
  "places" ... Done. (300 features)
  "pois" ... Done. (1107 features)
  "transport" ... Done. (65 features)
  "pofw" ... Done. (65 features)
  "natural" ... Done. (666 features)
  "railways" ... Done. (141 features)
  "roads" ... Done. (7425 features)
  "waterways" ... Done. (379 features)
  "protected_areas" ... Done. (20 features)
  "water" ... Done. (234 features)
  "landuse" ... Done. (2465 features)
  "buildings" ... Done. (5743 features)
  "adminareas" ... Done. (58 features)

>>> # Retrieve the data of specific layers
>>> layer_names = ['points', 'multipolygons']
>>> rutland_data_ = osmdb.fetch_data(subregion_name, layer_names, verbose=True)
Fetching the data of "Rutland" ...
  "points" ... Done.
  "multipolygons" ... Done.
>>> type(rutland_data_)
dict
>>> list(rutland_data_)
['points', 'multipolygons']

>>> # Data of the 'points' layer
>>> rutland_points = rutland_data_['points']
>>> rutland_points.head()
                                              points
0  {'type': 'Feature', 'geometry': {'type': 'Poin...
1  {'type': 'Feature', 'geometry': {'type': 'Poin...
2  {'type': 'Feature', 'geometry': {'type': 'Poin...
3  {'type': 'Feature', 'geometry': {'type': 'Poin...
4  {'type': 'Feature', 'geometry': {'type': 'Poin...

>>> # Retrieve the data of all the layers from the database
>>> rutland_data = osmdb.fetch_data(subregion_name, layer_names=None, verbose=True)
Fetching the data of "Rutland" ...
  "points" ... Done.
  "lines" ... Done.
  "multilinestrings" ... Done.
  "multipolygons" ... Done.
  "other_relations" ... Done.
  "adminareas" ... Done.
  "buildings" ... Done.
  "landuse" ... Done.
  "natural" ... Done.
  "places" ... Done.
  "pofw" ... Done.
  "pois" ... Done.
  "protected_areas" ... Done.
  "railways" ... Done.
  "roads" ... Done.
  "traffic" ... Done.
  "transport" ... Done.
  "water" ... Done.
  "waterways" ... Done.
>>> type(rutland_data)
dict
>>> list(rutland_data)
['points',
 'lines',
 'multilinestrings',
 'multipolygons',
 'other_relations',
 'adminareas',
 'buildings',
 'landuse',
 'natural',
 'places',
 'pofw',
 'pois',
 'protected_areas',
 'railways',
 'roads',
 'traffic',
 'transport',
 'water',
 'waterways']

>>> # Data of the 'waterways' layer
>>> rutland_waterways = rutland_data['waterways']
>>> rutland_waterways.head()
    osm_id  ...                                           geometry
0  3701346  ...  LINESTRING (-0.7536654 52.6495358, -0.7536236 ...
1  3701347  ...  LINESTRING (-0.7948821 52.6569468, -0.7946128 ...
2  3707149  ...  LINESTRING (-0.7262381 52.6790459, -0.7258244 ...
3  3707303  ...  LINESTRING (-0.7213277 52.6765954, -0.7206778 ...
4  4470795  ...  LINESTRING (-0.4995349 52.6418825, -0.4984075 ...
[5 rows x 6 columns]

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
Drop the database "osmdb_test" from postgres:***@localhost:5432?
 [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(data_dir, verbose=True)
Confirm deletion of the directory "tests/osm_data/" (Not empty)?
 [No]|Yes: yes
Deleting "tests/osm_data/" ... Done.