PostgresOSM.import_osm_layer¶
- PostgresOSM.import_osm_layer(layer_data, table_name, schema_name, table_named_as_subregion=False, schema_named_as_layer=False, if_exists='fail', chunk_size=None, confirmation_required=True, verbose=False, raise_error=True, **kwargs)[source]¶
Import one layer of OSM data into a table.
- Parameters:
layer_data (pandas.DataFrame | geopandas.GeoDataFrame) – one layer of OSM data
schema_name (str) – name of a schema (or name of a PBF layer)
table_name (str) – name of a table
table_named_as_subregion (bool) – whether to use subregion name as a table name. Defaults to
False.schema_named_as_layer (bool) – whether a schema is named as a layer name. Defaults to
False.if_exists (str) – if the table already exists. Valid options include
{'replace', 'append', 'fail'}. Defaults to'fail'.chunk_size (int | None) – the number of rows in each batch to be written at a time. Defaults to
None.confirmation_required (bool) – whether to prompt a message for confirmation to proceed. Defaults to
True.verbose (bool) – whether to print relevant information in console as the function runs. Defaults to
False.raise_error (bool) – Whether to raise the provided exception. If
raise_error=False, the error will be suppressed. Defaults toTrue.kwargs – Ooptional parameters of pyhelpers.dbms.PostgreSQL.import_data.
Examples:
>>> from pydriosm.ios._base import BaseIOS >>> from pyhelpers.dirs import delete_dir >>> osmdb = BaseIOS(database_name='osmdb_test', verbose=True) Password (postgres@localhost:5432): *** Creating a database: "osmdb_test" ... Done. Connecting postgres:***@localhost:5432/osmdb_test ... Successfully. >>> subrgn_name = 'Rutland' # name of a subregion >>> dat_dir = "tests/osm_data" # name of a data directory where the subregion data is
Example 1 - Import data of the ‘points’ layer of a PBF file:
>>> # First, read the PBF data of Rutland (from Geofabrik free download server) >>> # (If the data file is not available, it'll be downloaded by confirmation) >>> raw_pbf = osmdb.reader.read_pbf( ... subrgn_name, data_dir=dat_dir, download=True, verbose=True) Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.89M/1.89M | 5.76MB/s ... Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done. Reading "./tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done. >>> type(raw_pbf) dict >>> list(raw_pbf.keys()) ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations'] >>> # Get the data of 'points' layer >>> points_key = 'points' >>> raw_pbf_points = raw_pbf[points_key] >>> type(raw_pbf_points) list >>> type(raw_pbf_points[0]) osgeo.ogr.Feature >>> # Now import the data of 'points' into the PostgreSQL server >>> osmdb.import_osm_layer( ... layer_data=raw_pbf_points, table_name=subrgn_name, schema_name=points_key, ... verbose=True) To import data into the table "points"."Rutland" at postgres:***@localhost:5432/osm... ? [No]|Yes: yes Creating a schema: "points" ... Done. Importing the data into "points"."Rutland" ... Done. >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, points_key) >>> tbl_col_info.head() column_0 table_catalog osmdb_test table_schema points table_name Rutland column_name points ordinal_position 1 >>> # Parse the 'geometry' of the PBF data of Rutland >>> parsed_pbf = osmdb.reader.read_pbf( ... subregion_name=subrgn_name, data_dir=dat_dir, expand=True, ... parse_geometry=True, verbose=True) >>> type(parsed_pbf) dict >>> list(parsed_pbf.keys()) ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations'] >>> # Get the parsed data of 'points' layer >>> parsed_pbf_points = parsed_pbf[points_key] >>> type(parsed_pbf_points) pandas.DataFrame >>> parsed_pbf_points.head() id ... properties 0 488658 ... {'osm_id': '488658', 'name': 'Tickencote Inter... 1 13883868 ... {'osm_id': '13883868', 'name': None, 'barrier'... 2 14049101 ... {'osm_id': '14049101', 'name': None, 'barrier'... 3 14558402 ... {'osm_id': '14558402', 'name': None, 'barrier'... 4 14558409 ... {'osm_id': '14558409', 'name': None, 'barrier'... [5 rows x 3 columns] >>> # Import the parsed 'points' data into the PostgreSQL database >>> osmdb.import_osm_layer( ... layer_data=parsed_pbf_points, table_name=subrgn_name, schema_name=points_key, ... if_exists='replace', verbose=True) Import data into "points"."Rutland" at postgres:***@localhost:5432/osmdb_test? [No]|Yes: yes Importing the data ... Done. >>> # Get the information of the table "points"."Rutland" >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, points_key) >>> tbl_col_info.head() column_0 column_1 column_2 table_catalog osmdb_test osmdb_test osmdb_test table_schema points points points table_name Rutland Rutland Rutland column_name id geometry properties ordinal_position 1 2 3
Example 2 - Import data of the ‘railways’ layer of a GeoPackage*:
>>> # Read the data of 'railways' layer and delete the extracts >>> lyr_name = 'railways' >>> rutland_railways_gpkg = osmdb.reader.read_gpkg( ... subregion_name=subrgn_name, layer_names=lyr_name, data_dir=dat_dir, ... download=True, verbose=True) Downloading "rutland-latest-free.gpkg.zip" 100%|██████████| 3.30M/3.30M | 3.6... Saving "rutland-latest-free.gpkg.zip" to "./tests/osm_data/rutland/" ... Done. Parsing the data ... Done. >>> type(rutland_railways_gpkg) dict >>> list(rutland_railways_gpkg.keys()) ['railways'] >>> # Get the data of 'railways' layer >>> rutland_railways_gpkg_ = rutland_railways_gpkg[lyr_name] >>> rutland_railways_gpkg_.head() osm_id code ... tunnel geometry 0 2162114 6101 ... F LINESTRING (-0.46449 52.71043, -0.46111 52.706... 1 3681043 6101 ... F LINESTRING (-0.65312 52.57308, -0.65318 52.572... 2 3693985 6101 ... F LINESTRING (-0.72203 52.69658, -0.72182 52.697... 3 3693986 6101 ... F LINESTRING (-0.61731 52.61323, -0.62419 52.614... 4 8044108 6101 ... T LINESTRING (-0.69782 52.62858, -0.70271 52.63395) [5 rows x 8 columns] >>> # Import the 'railways' data into the PostgreSQL database >>> osmdb.import_osm_layer( ... rutland_railways_gpkg_, table_name=subrgn_name, schema_name=lyr_name, ... verbose=True) Import data into "railways"."Rutland" at postgres:***@localhost:5432/osmdb_test? [No]|Yes: yes Creating a schema: "railways" ... Done. Importing the data ... Done. >>> # Get the information of the table "railways"."Rutland" >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, lyr_name) >>> tbl_col_info.head() column_0 column_1 ... column_6 column_7 table_catalog osmdb_test osmdb_test ... osmdb_test osmdb_test table_schema railways railways ... railways railways table_name Rutland Rutland ... Rutland Rutland column_name osm_id code ... tunnel geometry ordinal_position 1 2 ... 7 8 [5 rows x 8 columns]
Delete the test database and downloaded data files:
>>> # Delete the database 'osmdb_test' >>> osmdb.drop_database(verbose=True) Drop the database "osmdb_test" from postgres:***@localhost:5432? [No]|Yes: yes Dropping "osmdb_test" ... Done. >>> # Delete the downloaded data files >>> delete_dir(dat_dir, verbose=True) To delete the directory "./tests/osm_data/" (Not empty) ? [No]|Yes: yes Deleting "./tests/osm_data/" ... Done.