PostgresOSM.import_osm_layer¶
- PostgresOSM.import_osm_layer(layer_data, table_name, schema_name, table_named_as_subregion=False, schema_named_as_layer=False, if_exists='fail', force_replace=False, chunk_size=None, confirmation_required=True, verbose=False, **kwargs)[source]¶
Import one layer of OSM data into a table.
- Parameters:
layer_data (pandas.DataFrame | geopandas.GeoDataFrame) – one layer of OSM data
schema_name (str) – name of a schema (or name of a PBF layer)
table_name (str) – name of a table
table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to
Falseschema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to
Falseif_exists (str) – if the table already exists, defaults to
'fail'; valid options include{'replace', 'append', 'fail'}force_replace (bool) – whether to force to replace existing table, defaults to
Falsechunk_size (int | None) – the number of rows in each batch to be written at a time, defaults to
Noneconfirmation_required (bool) – whether to prompt a message for confirmation to proceed, defaults to
Trueverbose (bool) – whether to print relevant information in console as the function runs, defaults to
Falsekwargs – [optional] parameters of pyhelpers.sql.PostgreSQL.dump_data()
Examples:
>>> from pydriosm.ios._base import BaseIOS >>> from pyhelpers.dirs import delete_dir >>> osmdb = BaseIOS(database_name='osmdb_test', verbose=True) Password (postgres@localhost:5432): *** Creating a database: "osmdb_test" ... Done. Connecting postgres:***@localhost:5432/osmdb_test ... Successfully. >>> subrgn_name = 'Rutland' # name of a subregion >>> dat_dir = "tests/osm_data" # name of a data directory where the subregion data is
Example 1 - Import data of the ‘points’ layer of a PBF file:
>>> # First, read the PBF data of Rutland (from Geofabrik free download server) >>> # (If the data file is not available, it'll be downloaded by confirmation) >>> raw_pbf = osmdb.reader.read_pbf( ... subrgn_name, data_dir=dat_dir, download=True, verbose=True) Downloading "rutland-latest.osm.pbf" 100%|██████████| 1.89M/1.89M | 5.76MB/s ... Saving "rutland-latest.osm.pbf" to "./tests/osm_data/rutland/" ... Done. Reading "./tests/osm_data/rutland/rutland-latest.osm.pbf" ... Done. >>> type(raw_pbf) dict >>> list(raw_pbf.keys()) ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations'] >>> # Get the data of 'points' layer >>> points_key = 'points' >>> raw_pbf_points = raw_pbf[points_key] >>> type(raw_pbf_points) list >>> type(raw_pbf_points[0]) osgeo.ogr.Feature >>> # Now import the data of 'points' into the PostgreSQL server >>> osmdb.import_osm_layer( ... layer_data=raw_pbf_points, table_name=subrgn_name, schema_name=points_key, ... verbose=True) To import data into the table "points"."Rutland" at postgres:***@localhost:5432/osm... ? [No]|Yes: yes Creating a schema: "points" ... Done. Importing the data into "points"."Rutland" ... Done. >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, points_key) >>> tbl_col_info.head() column_0 table_catalog osmdb_test table_schema points table_name Rutland column_name points ordinal_position 1 >>> # Parse the 'geometry' of the PBF data of Rutland >>> parsed_pbf = osmdb.reader.read_pbf( ... subregion_name=subrgn_name, data_dir=dat_dir, expand=True, ... parse_geometry=True) >>> type(parsed_pbf) dict >>> list(parsed_pbf.keys()) ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations'] >>> # Get the parsed data of 'points' layer >>> parsed_pbf_points = parsed_pbf[points_key] >>> type(parsed_pbf_points) pandas.DataFrame >>> parsed_pbf_points.head() id ... properties 0 488658 ... {'osm_id': '488658', 'name': 'Tickencote Inter... 1 13883868 ... {'osm_id': '13883868', 'name': None, 'barrier'... 2 14049101 ... {'osm_id': '14049101', 'name': None, 'barrier'... 3 14558402 ... {'osm_id': '14558402', 'name': None, 'barrier'... 4 14558409 ... {'osm_id': '14558409', 'name': None, 'barrier'... [5 rows x 3 columns] >>> # Import the parsed 'points' data into the PostgreSQL database >>> osmdb.import_osm_layer( ... layer_data=parsed_pbf_points, table_name=subrgn_name, schema_name=points_key, ... verbose=True, if_exists='replace') Proceed to import data into "points"."Rutland" at postgres:***@localhost:5432/osmdb_test ? [No]|Yes: yes The table "points"."Rutland" already exists and is replaced. Importing the data into the table "points"."Rutland" ... Done. >>> # Get the information of the table "points"."Rutland" >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, points_key) >>> tbl_col_info.head() column_0 column_1 column_2 table_catalog osmdb_test osmdb_test osmdb_test table_schema points points points table_name Rutland Rutland Rutland column_name id geometry properties ordinal_position 1 2 3
Example 2 - Import data of the ‘railways’ layer of a shapefile*:
>>> # Read the data of 'railways' layer and delete the extracts >>> lyr_name = 'railways' >>> rutland_railways_shp = osmdb.reader.read_shp( ... subregion_name=subrgn_name, layer_names=lyr_name, data_dir=dat_dir, ... download=True, rm_extracts=True, verbose=True) Downloading "rutland-latest-free.shp.zip" 100%|██████████| 2.71M/2.71M | 7.07... Saving "rutland-latest-free.shp.zip" to "./tests/osm_data/rutland/" ... Done. Extracting the following layer(s): 'railways' from: "./tests/osm_data/rutland/rutland-latest-free.shp.zip" ... to: "./tests/osm_data/rutland/rutland-latest-free-shp/" ... Done. Reading "./tests/osm_data/rutland/rutland-latest-free-shp/gis_osm_railways_free_1.s... Deleting the extracts "./tests/osm_data/rutland/rutland-latest-free-shp/" ... Done. >>> type(rutland_railways_shp) dict >>> list(rutland_railways_shp.keys()) ['railways'] >>> # Get the data of 'railways' layer >>> rutland_railways_shp_ = rutland_railways_shp[lyr_name] >>> rutland_railways_shp_.head() osm_id code ... coordinates shape_type 0 2162114 6101 ... [(-0.4644873, 52.7104315), (-0.4611095, 52.706... 3 1 3681043 6101 ... [(-0.6531215, 52.5730787), (-0.6531793, 52.572... 3 2 3693985 6101 ... [(-0.7220263, 52.696584), (-0.7218164, 52.6973... 3 3 3693986 6101 ... [(-0.6173072, 52.6132317), (-0.6241869, 52.614... 3 4 8044108 6101 ... [(-0.6978245, 52.6285831), (-0.7027126, 52.633... 3 [5 rows x 9 columns] >>> # Import the 'railways' data into the PostgreSQL database >>> osmdb.import_osm_layer( ... layer_data=rutland_railways_shp_, table_name=subrgn_name, schema_name=lyr_name, ... verbose=True) Proceed to import data into "railways"."Rutland" at postgres:***@localhost:5432/osmdb_test ? [No]|Yes: yes Creating a schema: "railways" ... Done. Importing the data into the table "railways"."Rutland" ... Done. >>> # Get the information of the table "railways"."Rutland" >>> tbl_col_info = osmdb.get_table_column_info(subrgn_name, lyr_name) >>> tbl_col_info.head() column_0 column_1 ... column_7 column_8 table_catalog osmdb_test osmdb_test ... osmdb_test osmdb_test table_schema railways railways ... railways railways table_name Rutland Rutland ... Rutland Rutland column_name osm_id code ... coordinates shape_type ordinal_position 1 2 ... 8 9 [5 rows x 9 columns]
Delete the test database and downloaded data files:
>>> # Delete the database 'osmdb_test' >>> osmdb.drop_database(verbose=True) To drop the database "osmdb_test" from postgres:***@localhost:5432 ? [No]|Yes: yes Dropping "osmdb_test" ... Done. >>> # Delete the downloaded data files >>> delete_dir(dat_dir, verbose=True) To delete the directory "./tests/osm_data/" (Not empty) ? [No]|Yes: yes Deleting "./tests/osm_data/" ... Done.