PostgresOSM.import_osm_layer¶
-
PostgresOSM.import_osm_layer(osm_layer_data, table_name, schema_name, table_named_as_subregion=False, schema_named_as_layer=False, if_exists='replace', force_replace=False, chunk_size=None, confirmation_required=True, verbose=False, **kwargs)¶ Import one layer of OSM data into a table.
- Parameters
osm_layer_data (pandas.DataFrame or geopandas.GeoDataFrame) – one layer of OSM data
schema_name (str) – name of a schema (or name of a PBF layer)
table_name (str) – name of a table
table_named_as_subregion (bool) – whether to use subregion name as a table name, defaults to
Falseschema_named_as_layer (bool) – whether a schema is named as a layer name, defaults to
Falseif_exists (str) – if the table already exists, to
'replace'(default),'append'or'fail'force_replace (bool) – whether to force to replace existing table, defaults to
Falsechunk_size (int or None) – the number of rows in each batch to be written at a time, defaults to
Noneconfirmation_required (bool) – whether to prompt a message for confirmation to proceed, defaults to
Trueverbose (bool) – whether to print relevant information in console as the function runs, defaults to
Falsekwargs – optional parameters of pyhelpers.sql.PostgreSQL.dump_data
Examples:
>>> import os >>> from pyhelpers.dir import cd >>> from pydriosm.ios import PostgresOSM >>> osmdb_test = PostgresOSM(database_name='osmdb_test') Password (postgres@localhost:5432): *** Connecting postgres:***@localhost:5432/osmdb_test ... Successfully. >>> sr_name = 'Rutland' # name of a subregion >>> dat_dir = "tests" # name of a data directory where the subregion data is >>> # -- Example 1: Import data of the 'points' layer of a PBF file -------------- >>> # First, read the PBF data of Rutland (from Geofabrik free download server) >>> # (If the data file is not available, it'll be downloaded by confirmation) >>> rutland_pbf_raw = osmdb_test.Reader.read_osm_pbf(sr_name, dat_dir, verbose=True) To download .pbf data of the following geographic region(s): London ? [No]|Yes: yes Downloading "rutland-latest.osm.pbf" to "tests\" ... Done. >>> # A quick view of the PBF data >>> type(rutland_pbf_raw) dict >>> list(rutland_pbf_raw.keys()) ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations'] >>> # Get the data of 'points' layer >>> rutland_pbf_raw_points = rutland_pbf_raw['points'] >>> rutland_pbf_raw_points.head() points 0 {"type": "Feature", "geometry": {"type": "Poin... 1 {"type": "Feature", "geometry": {"type": "Poin... 2 {"type": "Feature", "geometry": {"type": "Poin... 3 {"type": "Feature", "geometry": {"type": "Poin... 4 {"type": "Feature", "geometry": {"type": "Poin... >>> # Use the region name as a table name for storing the data in PostgreSQL server >>> tbl = sr_name >>> # Use a default layer (key) name as a schema name >>> schema = list(rutland_pbf_raw.keys())[0] # 'points' >>> # Now import the data of 'points' into the PostgreSQL server >>> osmdb_test.import_osm_layer(rutland_pbf_raw_points, tbl, schema, verbose=True) To import data into "points"."Rutland" at postgres:***@localhost:5432/osmdb_test ? [No]|Yes: yes Creating a schema: "points" ... Done. Importing the data into the table "points"."Rutland" ... Done. >>> tbl_col_info = osmdb_test.get_subregion_table_column_info(tbl, schema) >>> tbl_col_info.head() column_0 table_catalog osmdb_test table_schema points table_name Rutland column_name points ordinal_position 1 >>> rutland_pbf_parsed = osmdb_test.Reader.read_osm_pbf(sr_name, dat_dir, ... parse_raw_feat=True, ... transform_geom=True) >>> # Get the parsed data of 'points' layer >>> rutland_pbf_points = rutland_pbf_parsed[schema] >>> rutland_pbf_points.head() id ... other_tags 0 488432 ... "odbl"=>"clean" 1 488658 ... None 2 13883868 ... None 3 14049101 ... "traffic_calming"=>"cushion" 4 14558402 ... "direction"=>"clockwise" [5 rows x 12 columns] >>> # Import the parsed 'points' data into the PostgreSQL server >>> osmdb_test.import_osm_layer(rutland_pbf_points, tbl, schema, verbose=True) To import data into "points"."Rutland" at postgres:***@localhost:5432/osmdb_test ? [No]|Yes: yes The table "points"."Rutland" already exists and is replaced. Importing the data into the table "points"."Rutland" ... Done. >>> # Delete the downloaded PBF data file >>> os.remove(cd(dat_dir, "rutland-latest.osm.pbf")) >>> # -- Example 2: Import data of the 'railways' layer of a shapefile ----------- >>> # Read the data of 'railways' layer >>> # (without retaining any downloaded shapefile and extracts) >>> lyr_name = 'railways' >>> rutland_railways_shp = osmdb_test.Reader.read_shp_zip( ... subregion_name=sr_name, layer_names=lyr_name, data_dir=dat_dir, ... rm_extracts=True, rm_shp_zip=True, verbose=True) To download .shp.zip data of the following geographic region(s): Rutland ? [No]|Yes: yes Downloading "rutland-latest-free.shp.zip" to "tests\" ... Done. Extracting the following layer(s): 'railways' from "tests\rutland-latest-free.shp.zip" ... to "tests\rutland-latest-free-shp\" Done. Deleting the extracts "tests\rutland-latest-free-shp\" ... Done. Deleting "tests\rutland-latest-free.shp.zip" ... Done. >>> type(rutland_railways_shp) dict >>> list(rutland_railways_shp.keys()) # ['railways'] >>> # Get the data of 'railways' layer >>> rutland_railways_shp_ = rutland_railways_shp[lyr_name] >>> rutland_railways_shp_.head() osm_id code ... coordinates shape_type 0 2162114 6101 ... [(-0.4528083, 52.6993402), (-0.4518933, 52.698... 3 1 3681043 6101 ... [(-0.6531215, 52.5730787), (-0.6531793, 52.572... 3 2 3693985 6101 ... [(-0.7323403, 52.6782102), (-0.7319059, 52.678... 3 3 3693986 6101 ... [(-0.6173072, 52.6132317), (-0.6241869, 52.614... 3 4 4806329 6101 ... [(-0.4576926, 52.7035194), (-0.4565358, 52.702... 3 [5 rows x 9 columns] >>> osmdb_test.import_osm_layer(rutland_railways_shp_, table_name=sr_name, ... schema_name=lyr_name, verbose=True) To import data into "railways"."Rutland" at postgres:***@localhost:5432/osmdb_test ? [No]|Yes: yes Creating a schema: "railways" ... Done. Importing the data into the table "railways"."Rutland" ... Done. >>> tbl_col_info = osmdb_test.get_subregion_table_column_info(tbl, lyr_name) >>> tbl_col_info.head() column_0 column_1 ... column_7 column_8 table_catalog osmdb_test osmdb_test ... osmdb_test osmdb_test table_schema railways railways ... railways railways table_name Rutland Rutland ... Rutland Rutland column_name osm_id code ... coordinates shape_type ordinal_position 1 2 ... 8 9 [5 rows x 9 columns] >>> # Delete the database 'osmdb_test' >>> osmdb_test.drop_database(verbose=True) To drop the database "osmdb_test" from postgres:***@localhost:5432 ? [No]|Yes: yes Dropping "osmdb_test" ... Done.