PostgresOSM.fetch_osm_data

PostgresOSM.fetch_osm_data(subregion_name, layer_names=None, chunk_size=None, method='tempfile', max_size_spooled=1, decode_geojson=True, sort_by='id', table_named_as_subregion=False, schema_named_as_layer=False, verbose=False, **kwargs)[source]

Fetch OSM data (of one or multiple layers) of a geographic (sub)region.

See also [ROP-1].

Parameters:
  • subregion_name (str) – name of a geographic (sub)region (or the corresponding table)

  • layer_names (list | None) – names of schemas for each layer of the PBF data, if None (default), the default layer names as schema names

  • chunk_size (int | None) – the number of rows in each batch to be written at a time, defaults to None

  • method (str | None) – method to be used for buffering temporary data, defaults to 'tempfile'

  • max_size_spooled (int, float) – see pyhelpers.sql.PostgreSQL.read_sql_query(), defaults to 1 (in GB)

  • decode_geojson (bool) – whether to decode string GeoJSON data, defaults to True

  • sort_by (str | list) – column name(s) by which the data (fetched from PostgreSQL) is sorted, defaults to 'id'

  • table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to False

  • schema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to False

  • verbose (bool | int) – whether to print relevant information in console, defaults to False

Returns:

PBF (.osm.pbf) data

Return type:

dict | collections.OrderedDict

Examples:

>>> from pydriosm.ios import PostgresOSM
>>> from pyhelpers.dirs import delete_dir

>>> osmdb = PostgresOSM(database_name='osmdb_test')
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

>>> subrgn_name = 'Rutland'  # name of a subregion
>>> dat_dir = "tests\osm_data"  # name of a data directory where the subregion data is

>>> # Import PBF data of Rutland
>>> osmdb.import_subregion_osm_pbf(subrgn_name, data_dir=dat_dir, verbose=True)
To import .osm.pbf data of the following geographic (sub)region(s):
    "Rutland"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "rutland-latest.osm.pbf"
    to "tests\osm_data\rutland\" ... Done.
Reading "tests\osm_data\rutland\rutland-latest.osm.pbf" ... Done.
Importing the data into table "Rutland" ...
    "points" ... Done. (<total of rows> features)
    "lines" ... Done. (<total of rows> features)
    "multilinestrings" ... Done. (<total of rows> features)
    "multipolygons" ... Done. (<total of rows> features)
    "other_relations" ... Done. (<total of rows> features)

>>> # Import shapefile data of Rutland
>>> rutland_shp = osmdb.reader.read_shp_zip(
...     subrgn_name, data_dir=dat_dir, rm_extracts=True, verbose=True)
Downloading "rutland-latest-free.shp.zip"
    to "tests\osm_data\rutland\" ... Done.
Extracting "tests\osm_data\rutland\rutland-latest-free.shp.zip"
    to "tests\osm_data\rutland\rutland-latest-free-shp\" ... Done.
Reading the shapefile(s) at
    "tests\osm_data\rutland\rutland-latest-free-shp\" ... Done.
Deleting the extracts "tests\osm_data\rutland\rutland-latest-free-shp\" ... Done.
>>> osmdb.import_osm_data(rutland_shp, table_name=subrgn_name, verbose=True)
To import data into table "Rutland" at postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Importing the data ...
    "buildings" ... Done. (<total of rows> features)
    "landuse" ... Done. (<total of rows> features)
    "natural" ... Done. (<total of rows> features)
    "places" ... Done. (<total of rows> features)
    "pofw" ... Done. (<total of rows> features)
    "pois" ... Done. (<total of rows> features)
    "railways" ... Done. (<total of rows> features)
    "roads" ... Done. (<total of rows> features)
    "traffic" ... Done. (<total of rows> features)
    "transport" ... Done. (<total of rows> features)
    "water" ... Done. (<total of rows> features)
    "waterways" ... Done. (<total of rows> features)

>>> # Retrieve the data of specific layers
>>> lyr_names = ['points', 'multipolygons']
>>> rutland_data_ = osmdb.fetch_osm_data(subrgn_name, lyr_names, verbose=True)
Fetching the data of "Rutland" ...
    "points" ... Done.
    "multipolygons" ... Done.
>>> type(rutland_data_)
collections.OrderedDict
>>> list(rutland_data_.keys())
['points', 'multipolygons']

>>> # Data of the 'points' layer
>>> rutland_points = rutland_data_['points']
>>> rutland_points.head()
                                              points
0  {'type': 'Feature', 'geometry': {'type': 'Poin...
1  {'type': 'Feature', 'geometry': {'type': 'Poin...
2  {'type': 'Feature', 'geometry': {'type': 'Poin...
3  {'type': 'Feature', 'geometry': {'type': 'Poin...
4  {'type': 'Feature', 'geometry': {'type': 'Poin...

>>> # Retrieve the data of all the layers from the database
>>> rutland_data = osmdb.fetch_osm_data(subrgn_name, layer_names=None, verbose=True)
Fetching the data of "Rutland" ...
    "points" ... Done.
    "lines" ... Done.
    "multilinestrings" ... Done.
    "multipolygons" ... Done.
    "other_relations" ... Done.
    "buildings" ... Done.
    "landuse" ... Done.
    "natural" ... Done.
    "places" ... Done.
    "pofw" ... Done.
    "pois" ... Done.
    "railways" ... Done.
    "roads" ... Done.
    "traffic" ... Done.
    "transport" ... Done.
    "water" ... Done.
    "waterways" ... Done.
>>> type(rutland_data)
collections.OrderedDict
>>> list(rutland_data.keys())
['points',
 'lines',
 'multilinestrings',
 'multipolygons',
 'other_relations',
 'buildings',
 'landuse',
 'natural',
 'places',
 'pofw',
 'pois',
 'railways',
 'roads',
 'traffic',
 'transport',
 'water',
 'waterways']

>>> # Data of the 'waterways' layer
>>> rutland_waterways = rutland_data['waterways']
>>> rutland_waterways.head()
    osm_id  code  ...                                        coordinates  shape_type
0  3701346  8102  ...  [(-0.7536654, 52.6495358), (-0.7536236, 52.649...           3
1  3701347  8102  ...  [(-0.7948821, 52.6569468), (-0.7946128, 52.656...           3
2  3707149  8103  ...  [(-0.7262381, 52.6790459), (-0.7258244, 52.680...           3
3  3707303  8102  ...  [(-0.7213277, 52.6765954), (-0.7206778, 52.676...           3
4  4470795  8101  ...  [(-0.4995349, 52.6418825), (-0.4984075, 52.642...           3
[5 rows x 7 columns]

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
To drop the database "osmdb_test" from postgres:***@localhost:5432
? [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(dat_dir, verbose=True)
To delete the directory "tests\osm_data\" (Not empty)
? [No]|Yes: yes
Deleting "tests\osm_data\" ... Done.

See also