PostgresOSM.import_subregion_osm_pbf

PostgresOSM.import_subregion_osm_pbf(subregion_names, data_dir=None, update_osm_pbf=False, if_exists='fail', chunk_size_limit=50, expand=False, parse_geometry=False, parse_properties=False, parse_other_tags=False, pickle_pbf_file=False, rm_pbf_file=False, confirmation_required=True, verbose=False, **kwargs)[source]

Import data of geographic (sub)region(s) that do not have (sub-)subregions into a database.

Parameters:

subregion_names (str | list | None) – name(s) of geographic (sub)region(s)
data_dir (str | None) – directory where the PBF data file is located/saved; if None (default), the default directory
update_osm_pbf (bool) – whether to update .osm.pbf data file (if available), defaults to False
if_exists (str) – if the table already exists, defaults to 'fail'; valid options include {'replace', 'append', 'fail'}
chunk_size_limit (int) – threshold (in MB) that triggers the use of chunk parser, defaults to 50; if the size of the .osm.pbf file (in MB) is greater than chunk_size_limit, it will be parsed in a chunk-wise way
expand (bool) – whether to expand dict-like data into separate columns, defaults to False
parse_geometry (bool) – whether to represent the 'geometry' field in a shapely.geometry format, defaults to False
parse_properties (bool) – whether to represent the 'properties' field in a tabular format, defaults to False
parse_other_tags (bool) – whether to represent a 'other_tags' (of 'properties') in a dict format, defaults to False
pickle_pbf_file (bool) – whether to save the .pbf data as a .pickle file, defaults to False
rm_pbf_file (bool) – whether to delete the downloaded .osm.pbf file, defaults to False
confirmation_required (bool) – whether to ask for confirmation to proceed, defaults to True
verbose (bool | int) – whether to print relevant information in console, defaults to False
kwargs – [optional] parameters of the method _import_subregion_osm_pbf() or _import_subregion_osm_pbf_chunk_wisely()

Examples:

>>> from pydriosm.ios import PostgresOSM
>>> from pyhelpers.dirs import cd, delete_dir
>>> from pyhelpers.store import load_pickle

>>> osmdb = PostgresOSM(database_name='osmdb_test')
Password (postgres@localhost:5432): ***
Creating a database: "osmdb_test" ... Done.
Connecting postgres:***@localhost:5432/osmdb_test ... Successfully.

Example 1 - Import PBF data of Rutland:

>>> subrgn_name = 'Rutland'  # name of a subregion
>>> dat_dir = "tests\osm_data"  # name of a data directory where the subregion data is

>>> osmdb.import_subregion_osm_pbf(subrgn_name, data_dir=dat_dir, verbose=True)
To import .osm.pbf data of the following geographic (sub)region(s):
    "Rutland"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "rutland-latest.osm.pbf"
    to "tests\osm_data\rutland" ... Done.
Reading "tests\osm_data\rutland\rutland-latest.osm.pbf" ... Done.
Importing the data into table "Rutland" ...
    "points" ... Done. (<total of rows> features)
    "lines" ... Done. (<total of rows> features)
    "multilinestrings" ... Done. (<total of rows> features)
    "multipolygons" ... Done. (<total of rows> features)
    "other_relations" ... Done. (<total of rows> features)

Example 2 - Import PBF data of Leeds and London:

>>> # Change the data source
>>> osmdb.data_source = 'BBBike'
>>> subrgn_names = ['Leeds', 'London']

>>> # Note this may take a few minutes (or longer)
>>> osmdb.import_subregion_osm_pbf(
...     subregion_names=subrgn_names, data_dir=dat_dir, expand=True,
...     parse_geometry=True, parse_properties=True, parse_other_tags=True,
...     pickle_pbf_file=True, rm_pbf_file=True, verbose=True)
To import .osm.pbf data of the following geographic (sub)region(s):
    "Leeds"
    "London"
  into postgres:***@localhost:5432/osmdb_test
? [No]|Yes: yes
Downloading "Leeds.osm.pbf"
    to "tests\osm_data\leeds\" ... Done.
Reading "tests\osm_data\leeds\Leeds.osm.pbf" ... Done.
Importing the data into table "Leeds" ...
    "points" ... Done. (82137 features)
    "lines" ... Done. (164411 features)
    "multilinestrings" ... Done. (390 features)
    "multipolygons" ... Done. (439144 features)
    "other_relations" ... Done. (6938 features)
Saving "Leeds-pbf.pickle" to "tests\osm_data\leeds\" ... Done.
Deleting "tests\osm_data\leeds\Leeds.osm.pbf" ... Done.
Downloading "London.osm.pbf"
    to "tests\osm_data\london\" ... Done.
Importing the data of "London" chunk-wisely
  into postgres:***@localhost:5432/osmdb_test ...
    "points" ... Done. (654517 features)
    "lines" ... Done. (769631 features)
    "multilinestrings" ... Done. (7241 features)
    "multipolygons" ... Done. (5432 features)
    "other_relations" ... Done. (21792 features)
Saving "London-pbf.pickle" to "tests\osm_data\london\" ... Done.
Deleting "tests\osm_data\london\London.osm.pbf" ... Done.

>>> # As `pickle_pbf_file=True`, the parsed PBF data have been saved as pickle files

>>> # Data of Leeds
>>> leeds_pbf = load_pickle(cd(dat_dir, "leeds", "Leeds-pbf.pickle"))
>>> type(leeds_pbf)
dict
>>> list(leeds_pbf.keys())
['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']
>>> # Data of the 'points' layer of Leeds
>>> leeds_pbf_points = leeds_pbf['points']
>>> leeds_pbf_points.head()
       id                       geometry  ... man_made             other_tags
0  154941  POINT (-1.5560511 53.6879848)  ...     None                   None
1  154962     POINT (-1.34293 53.844618)  ...     None  {'name:signed': 'no'}
2  155014   POINT (-1.517335 53.7499667)  ...     None  {'name:signed': 'no'}
3  155023   POINT (-1.514124 53.7416937)  ...     None  {'name:signed': 'no'}
4  155035   POINT (-1.516511 53.7256632)  ...     None  {'name:signed': 'no'}
[5 rows x 11 columns]

>>> # Data of London
>>> london_pbf = load_pickle(cd(dat_dir, "london", "London-pbf.pickle"))
>>> type(london_pbf)
dict
>>> list(london_pbf.keys())
['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']
>>> # Data of the 'points' layer of London
>>> london_pbf_points = london_pbf['points']
>>> london_pbf_points.head()
      id  ...                                         other_tags
0  99878  ...  {'access': 'permissive', 'bicycle': 'no', 'mot...
1  99880  ...  {'crossing': 'unmarked', 'crossing:island': 'n...
2  99884  ...                        {'amenity': 'waste_basket'}
3  99918  ...                         {'emergency': 'life_ring'}
4  99939  ...           {'traffic_signals:direction': 'forward'}
[5 rows x 11 columns]

Delete the test database and downloaded data files:

>>> # Delete the database 'osmdb_test'
>>> osmdb.drop_database(verbose=True)
To drop the database "osmdb_test" from postgres:***@localhost:5432
? [No]|Yes: yes
Dropping "osmdb_test" ... Done.

>>> # Delete the downloaded data files
>>> delete_dir(dat_dir, verbose=True)
To delete the directory "tests\osm_data\" (Not empty)
? [No]|Yes: yes
Deleting "tests\osm_data\" ... Done.