mirror of
https://github.com/opengeos/GeoLibre.git
synced 2026-11-04 10:43:36 +00:00
main
33
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Qiusheng Wu
|
409e498102 |
fix(postgis): lift editable database layers past the 50k feature cap (#3272)
* fix(postgis): lift editable database layers past the 50k feature cap Editable PostGIS and SQL Server layers reused vector_ops.MAX_FEATURES, the 50,000-feature cap meant for geoprocessing payloads, so any table over 50k rows failed to load with "Table exceeds the 50000-feature limit for editable layers". Give database layers their own cap, 1,000,000 by default and set with GEOLIBRE_DB_MAX_FEATURES on the processing server. Reading a 1.2M-point table takes about ten seconds. The 413 message now names that setting, and for PostGIS points to the read-only vector tiles mode. Fixes #3270 * fix(postgis): join the cap message explicitly and pin the cap in tests Build the 413 detail from its parts instead of relying on a trailing space in the hint, share it with require_features so the write path has one wording, and pin GEOLIBRE_DB_MAX_FEATURES in the PostGIS tests so a value exported in the shell cannot change their results. * fix(postgis): bound the cap to bigint and cover the SQL Server paths Reject GEOLIBRE_DB_MAX_FEATURES values outside the bigint range, since the cap is bound into LIMIT / TOP (?). Add SQL Server read and write-back cap tests, and note the per-read memory cost in the self-hosting docs. |
||
 Thang Quach
|
4ca5e7dd49 |
Add MBTiles Layer support on GeoLibre Web (#3185)
* Add MBTiles Layer support on GeoLibre Web * fix(mbtiles): enforce 1 GB browser limit and free databases when layers are removed * fix(mbtiles): make browser database keys unique across sessions |
||
|
Qiusheng Wu
|
839a24dd86 |
feat: ICESat-2 ATL03 photons, read lazily for the map view (#3091)
* feat: ICESat-2 ATL03 photons, read lazily for the map view ATL03 granules hold every photon of an orbit segment (1-7 GB), so they are never loaded whole. A Web Worker opens the granule lazily and reads only what the map view needs: each beam's 20 m geolocation segments (found by a binary search on latitude, shared across the six beams), then just the photon index range inside the view. - Local files: the dialog picks a File (a file input in every build) and the worker mounts it with WORKERFS, so reads are FileReaderSync slices. - NASA granules: Earthaccess "Open" on an ATL03 granule hands the dialog the Earthdata relay URL; the worker reads it with synchronous ranged GETs (custom lazy device: size from a ranged GET, the bearer token, exact reads for whole chunks, an LRU block cache), split across up to six workers by beam, with live MB progress. Photons are merged and thinned evenly once. - Add Data → ICESat-2 / GEDI shows ground tracks with photon counts, surface type, lowest signal confidence and a photon cap; the layer carries h_ph, signal_conf, time and distance_km and is tagged for the along-track profile, which gains a "Photon heights" dot scatter. - Matches a brute-force h5py scan of every photon exactly (per beam) on two real granules (264 MB and 528 MB). Also follow-ups from #3087 / #3089 review: an integer dataset's _FillValue is read as an exact int (uint64 shot numbers), and the profile resets a beam or preset a layer edit made invalid and uses useId() for its clip path. Part of #3078 (Phase 2: ATL03 photon-level data). * Address review feedback - atl03-client: drop a helper worker that fails to start (handled, so no unhandled rejection) so the next read retries it; each worker caps its own part at maxPoints and the merge thins over what arrived, so a dense view does not clone millions of photons only to discard them. - atl03.ts: refuse a server that ignores Range for any start (the probe at 0 included) instead of reading a granule whole; keep ~32 MB of blocks a worker; accept views reported unwrapped across the antimeridian (east > 180). - atl03.worker: clear the previous file and disposer on re-open. - SpaceborneAtl03Options: measure an antimeridian-crossing view's width correctly, and keep existing layer metadata when tagging the layer. - AddSpaceborneLidarDialog: attach the file input while picking. - docs: list ATL03 among the popular datasets. - spaceborne_lidar.py: ignore a malformed integer _FillValue instead of failing the read. * Address review feedback - atl03-client: refuse a read after close() up front, and close helpers spawned during a read that close() interrupted. - AddSpaceborneLidarDialog: clear a stale error or status when the Earthaccess handoff opens an ATL03 granule. - atl03.ts: never read the segment arrays past the dataset's end when the granule's last segment is in view; correct the merge's doc. |
||
|
Qiusheng Wu
|
35facfacbf |
feat: along-track profile for ICESat-2 / GEDI footprint layers (#3089)
* feat: along-track profile for ICESat-2 / GEDI footprint layers Add Layer actions → Along-track profile for footprint layers: a movable, resizable window that charts distance along one beam against ground and canopy top (ICESat-2 ATL08: h_te_best_fit and + h_canopy; GEDI L2A / L2B: elev_lowestmode and + rh98 / rh100), the surface height (ATL06), or up to three numeric fields. - Linked to the map both ways: hovering drops a marker on the footprint, clicking selects it (standard map highlight), and a footprint selected on the map or in the attribute table is marked on the chart, switching beams when needed. - Wheel zooms along the track around the cursor, drag pans, double-click resets; lines break at missing values and real gaps (over five times the typical spacing); PNG and CSV export. - New footprint layers are tagged metadata.sourceKind "spaceborne-lidar" with the product and beams, and features get a stable id so map selection matches them exactly (the Python helpers in #3087 do the same). Untagged layers whose points carry beam and distance_km are recognized too. Part of #3078 (Phase 2: along-track profile view). * Address review feedback - AlongTrackProfileWindow: a click or tap selects the footprint nearest the pointer from the event itself, so touch and pen taps (which never set a hover) select too; one helper serves hover and click. - AlongTrackProfileWindow: reset the zoom only when the beam or the series keys change, so a layer edit such as a restyle keeps the zoom. - AddSpaceborneLidarDialog: keep any metadata the store set when tagging the layer. * Address review feedback - AlongTrackProfileWindow: follow a selected footprint's beam only when the selection changes, so the beam dropdown works while a footprint stays selected (the effect no longer depends on the beam). - AlongTrackProfileWindow: hover and click pick a footprint only within 16 px of the pointer, so a click inside a plotted gap selects nothing instead of a distant footprint. |
||
|
Qiusheng Wu
|
b606f13441 |
feat(plugins): NASA Earthaccess plugin for CMR search and Earthdata download (#3085)
* feat(plugins): add NASA Earthaccess plugin for CMR search and download Add a Web Services plugin that searches NASA's Common Metadata Repository for datasets and granules over the map view and a date range, draws granule footprints as a Layers-panel entry, and downloads granules with an Earthdata Login token. ICESat-2 (ATL06, ATL08) and GEDI (L2A, L2B, L4A) granules open straight into Add Data -> ICESat-2 / GEDI. The DAACs send no CORS headers and refuse a preflighted Authorization header, so downloads need a non-browser hop: - Desktop: a new native download_remote_file command streams the file to disk (no 64 MiB cap, progress over a channel, cancellable), either to a path picked in a native save dialog or to the app cache, read back once by take_cached_download. JavaScript never names a write path. Exposed to plugins as app.downloadRemoteFile. - Web: a new /earthdata/download route on the tiles Worker, origin gated, that accepts only NASA Earthdata data hosts, sends the bearer token to the first hop only, follows the presigned CloudFront/S3 redirect, and streams the file back with CORS and no-store. Desktop users can also sign in with a user name and password (via nativeFetch, since the EDL token endpoint has no CORS); only the returned token is kept, in the keychain through app.credentials. Part of #3078 (Phase 2: granule search and authenticated download). * fix(earthaccess): Earthdata Search details link, CI checks CMR serves granule records only as metadata formats, so Details now opens the granule in Earthdata Search. Also add the plugin to the generated plugin reference, list downloadRemoteFile as a desktop-only app API member, and typecheck the Worker proxy test with the Workers types. * Address Claude review feedback - earthdata.ts: catch a malformed redirect Location and answer 502 with CORS instead of an uncaught 500, and narrow the redirect allowlist from all of amazonaws.com to S3 endpoints. - remote_download.rs: request Accept-Encoding: identity (and keep callers from overriding it) so the Content-Length check compares like with like. - remote_download.rs: reject a reused request ID before the save dialog opens, and never let a new cache entry orphan an earlier one's file. * feat(earthaccess): multi-file download and NASA COG map layers - Multi-file granules (an HLS scene has 15 files) get a collapsible file list with a Download per file, and Download all: the desktop app asks for a folder once (new pick_download_folder command; JavaScript holds only the folder's id, never a path), the browser saves each file. - GeoTIFFs in the list get Add, which puts them on the map as COG layers. The COG readers fetch with the global fetch and take no headers, and the DAACs send no CORS headers, so a layer reads through the tiles Worker relay. Its saved URL carries no token; a fetch wrapper installed at boot adds the Earthdata Login token to requests for that exact route, so reopened projects keep working. - The Worker caches presigned redirect targets (keyed by a hash of the token and URL, in memory and in the edge cache, 50 minutes), cutting a cached range request from about 1.9 s to about 0.1 s. - File names in the list drop the shared granule prefix (B04.tif), and toggling the list no longer closes it by re-rendering the card. * fix(earthaccess): refresh Add/Added when a COG layer is removed The panel computed a file's Added state only when it re-rendered, so removing the layer from the Layers panel left the button disabled. The panel now re-renders when the set of Earthdata COG layers in the store changes, and ignores unrelated layer edits. * Address review feedback - earthdata.ts: restrict the content-disposition file name to [A-Za-z0-9._-] so an odd URL path cannot make Headers.set throw, and request identity encoding upstream so Content-Length matches the body. - spaceborne-lidar-handoff.ts: isolate listener failures so one bad listener cannot stop the others or reach the plugin. - maplibre-earthaccess.ts: fill MultiPolygon footprints too (CMR reports several polygons or boxes for some granules). - maplibre-earthaccess.ts: freeze the search's date range so Load more pages the same query after the date inputs are edited. - maplibre-earthaccess.ts: give the token link an href so keyboard users can reach it. - remote_download.rs: a folder download never replaces an existing file (adds " (n)"), and folder ids come from a counter. * Address Claude review feedback - maplibre-earthaccess.ts / earthdata-relay.ts: send the Earthdata token only to HTTPS NASA Earthdata data hosts (isEarthdataDataUrl). CMR data links are chosen by each provider, so a link on another host gets no token on desktop, has no Add, and opens in a new tab on the web (the relay serves Earthdata hosts only). - remote_download.rs: accept only Authorization, Cookie and Accept headers (the ones reqwest strips on a cross-host redirect, plus Accept), and refuse to send them over plain HTTP; the module docs now say exactly which headers stay on the first host. - earthdata-fetch-auth.ts: install the fetch wrapper through an explicit installEarthdataFetchAuth() call in main.tsx, placed right before installDiagnosticsCapture(), instead of an import side effect. * Address review feedback - remote_download.rs: requests that carry credentials use a client whose redirects must stay on HTTPS, since reqwest strips credentials only when the host changes (a same-host https -> http hop would resend them). - remote_download.rs: a folder download reserves its destination with create_new before transferring (name, then "name (n)"), streams into a request-specific .part file, and removes the placeholder if it fails, so concurrent downloads of one name never collide or replace a file. - types.ts / remote-download.ts: GeoLibreRemoteDownloadOptions is a discriminated union that requires folderId for "folder", and the host rejects a missing id at runtime. - maplibre-earthaccess.ts: the browser's Download all reports that the downloads started and mentions the multiple-downloads prompt, instead of claiming every file was saved. - earthdata.ts: edge-cache hits go through rememberPresigned so the in-memory cap applies. * Address review feedback - earthdata.ts: cache the first redirect only when it is a presigned target, not another Earthdata host that would fail the tokenless replay and be evicted on every request. - remote_download.rs: report a panicked transfer task as a failure, not as "cancelled". - remote_download.rs: create the folder placeholder's cleanup guard right after the reservation and move it into the task, so an abort before the task first runs still removes the empty file. |
||
|
Qiusheng Wu
|
b01611255e |
feat(data): add ICESat-2 and GEDI spaceborne LiDAR footprints (#3079)
* feat(data): add ICESat-2 and GEDI spaceborne LiDAR footprints Add Data -> ICESat-2 / GEDI reads a local HDF5 granule of ICESat-2 ATL06/ATL08 or GEDI L2A/L2B/L4A in the browser (h5wasm) and adds its footprints as a GeoJSON point layer. The NetCDF reader skipped these products because their per-footprint arrays are 1-D. The reader picks a product description from the root short_name (or the DAAC file name), lists beams and every 1-D field aligned with the footprints, applies the product's quality flag and fill values, and adds beam, beam type, UTC time and along-track distance to each feature. It can limit the read to the current view and thins every beam evenly to a point cap. The new layer is colored by the product's main field, so the attribute table charts give an along-track profile. Refs #3078 * Address review feedback - Thin footprints with one stride over all beams' matches, so maxPoints caps the total; restarting the index per beam kept the first match of every beam and could exceed the cap (CodeRabbit). - Null 64-bit integer fields that equal their _FillValue, compared as BigInt so the fill is not lost to float64 rounding (Claude). - Fit a track that crosses the antimeridian with unwrapped bounds instead of zooming out to the whole world; pointBounds moves to lib/ so it can be unit-tested (Claude). * Address review feedback - Fall back to the default point cap when the Maximum footprints box is empty or invalid, instead of lifting the cap and building a million-feature layer from a full GEDI orbit (Claude). - Compare a numeric 64-bit _FillValue only when it is a safe integer, as a BigInt, so a rounded fill cannot null a neighboring shot number; test added (CodeRabbit). * Address Claude review feedback - Drop an Add whose granule was closed during the pre-read yield (dialog closed or another file picked), instead of reading the closed file and reporting "No footprints matched" for the wrong granule. * Address Claude review feedback - Say what the quality filter does for each product under the checkbox, and relabel it "Apply the product's quality filter": ATL08 has no quality flag and only drops segments without a terrain height, so "pass the product's quality flag" overstated it. - Drop the unreachable shift in pointBounds: the shifted range is only narrower when points straddle 180°, which keeps west360 in [0, 180]. * Address CodeRabbit review feedback - Clear the busy state when another file is picked: a submit superseded during its pre-read yield returns early without clearing it, which left the Add button disabled for the newly opened granule. * Address Claude review feedback - Clamp a fractional maxPoints to at least 1, so 0.5 no longer floors to 0 and yields an infinite stride. - Keep the first field with a name and rename only later duplicates to their path, so a product's default names (and the color-by primary field) stay stable when another dataset shares a name. * feat(data): ICESat-2/GEDI sample granules, resizable dialog, Files section - Add a Sample data dropdown with one granule per product (ATL06, ATL08, GEDI L2A/L2B/L4A) hosted on Source Cooperative, downloaded with a progress readout into a preallocated buffer and cancelled when the dialog closes or another file is picked. - Stop the horizontal scrollbar: the dialog body is a grid whose column grew to the longest unbreakable HDF5 path (ATL06), so the form gets min-w-0 and field paths truncate. - Widen the default dialog and add a corner resize grip, through a useDialogResize hook that follows the layer metadata dialog's grip. - Move the menu entry from 3D layers to Files: the footprints render as flat points read from a granule file. * Split the ICESat-2/GEDI dialog and handle encoded sample downloads - check:untested-modules failed: the dialog reached 615 lines with no test loading it. Move the sample list, downloadWithProgress, fieldKey and baseName into lib/spaceborne-lidar-samples.ts (with tests), and the beam/field/filter controls into SpaceborneLidarOptions.tsx; the dialog is now 435 lines. - Read a content-encoded response whole instead of preallocating from Content-Length, which is the compressed size there and would overflow the buffer (Claude review). |
||
|
Qiusheng Wu
|
38ed979d3b |
docs: review the docs against the code and fix drift (#3008)
* docs: review the docs against the code and fix drift Check every docs page against the current code and correct what no longer matches: menu paths and labels, plugin placement (most plugins now open in a docked panel), Cesium ion tokens being optional, desktop OAuth sign-in for sharing, browser project save/open, project schema fields, server and collaboration endpoints, Python and MCP APIs, and stale paths in the source-code deep dives. Fix the Spatial SQL tutorial's ST_Area_Spheroid query, which needs ST_FlipCoordinates for lat/lon axis order. Document the Settings sections that were listed but never described (Language, Appearance, Interface, Geocoding, AI Providers), and add the USGS NLDI and ArcGIS feature editing pages to the nav. * docs: keep the NLDI and ArcGIS editing pages out of the nav * Address CodeRabbit review feedback - collaboration.md: give both snapshot-cap defaults, 10 MB for the relay and 1 MB in the Docker Compose setup. - features.md, cesium-renderer.md: without an Ion token the globe still gets terrain from keyless Terrarium elevation (or a configured COG), not a bare ellipsoid (cesium-engine.ts enableWorldTerrain). * Address CodeRabbit review feedback - project-format.md: type blankBackgroundColor and selectedLayerId as string | null, matching ProjectFile in packages/core/src/types.ts. |
||
|
Qiusheng Wu
|
d911a47978 |
feat(add-data): add administrative boundaries from geoBoundaries (#2999)
* feat(add-data): add administrative boundaries from geoBoundaries Add Data > Web Services > Administrative Boundaries loads a country's admin boundaries (ADM0 to ADM5, depending on the country) from the geoBoundaries gbOpen release, in simplified or full geometry. geoBoundaries is used instead of GADM: its API and LFS media host send CORS headers, so the dialog works in the desktop, web, and Jupyter builds, and its boundaries carry open licenses. GADM sends no CORS headers and does not allow redistribution. Refs discussion #2997. * Address review feedback - Keep the country-list error separate from the level error, show a "Country list unavailable" placeholder, and add a Retry button so a failed list is not a permanently disabled select (Claude). - Add request timeouts, abort an in-flight boundary download when the dialog closes, and note under the unchecked simplified option that full geometry is saved in the project file (Claude). - Turn a non-JSON 200 body into a translated "not valid GeoJSON" error and an HTTP error into the translated "Request failed with status" message (CodeRabbit). - Rewrite only wmgeolab/geoBoundaries github raw links to the LFS media host (CodeRabbit). * Address CodeRabbit review feedback - Read the response body as text before parsing, so a body-stream transport failure is reported as a request failure and only a JSON parse failure becomes the invalid-data error. |
||
 Daniel Hofbauer
|
425ac2c494 |
feat(browser): forget saved PostgreSQL connections (#2972)
* feat(browser): forget saved PostgreSQL connections * fix(browser): isolate PostgreSQL forget outcomes * fix(browser): invalidate forgotten PostgreSQL requests |
||
|
Daniel Hofbauer
|
6173e4835a | feat(browser): group PostgreSQL and SQL Server under Databases (#2929) (#2968) | ||
|
Qiusheng Wu
|
2baa7578dc |
feat(wfs): load GML from WFS servers without GeoJSON output (#2747)
* feat(wfs): load GML from WFS servers without GeoJSON output Many WFS servers (MapServer without an OGR output format, most INSPIRE services) answer GetFeature only in GML, so the GeoJSON-only loader failed with "returned XML instead of GeoJSON" (#2746). - Add lib/gml.ts, an in-browser GML 2 / 3.1 / 3.2 feature-collection parser: points, lines, polygons, their multi forms, and curves and surfaces built from linear segments, matched by local name so every GML/WFS namespace version works. - Normalize coordinates to lon/lat: URN and opengis.net/def CRS names follow EPSG axis order (urn:ogc:def:crs:EPSG::4326 is lat/lon), Web Mercator is unprojected, and any other CRS is rejected with a message naming it rather than drawn in the wrong place. - Type attribute columns only when every value qualifies, so a zero-padded code column ("08", "32") stays uniformly text. - fetchWfsGeoJson tries the version's default GML format and then no outputFormat after the GeoJSON aliases, with a fresh timeout budget for the (much larger) GML download. A GML feature collection is accepted for any request, which also makes layer refresh work. - Mention GML in the WFS dialog description in every locale and in the docs. * fix(wfs): reproject GML, fix ArcGIS axis order, bypass CORS on desktop Follow-ups to the GML support for #2746: - Reproject GML in any EPSG CRS (lib/gml-projection.ts) from the bundled EPSG tables, now shared with WMS tile reprojection through lib/epsg-proj4.ts. MapServer's WFS 1.0.0 ignores srsName and answers in its national grid (EPSG:2180 for the Polish PRG service). - Apply EPSG axis order only to GML 3 positions. GML 2 coordinates are always x, y: MapServer labels EPSG:2180 with a north-first URN yet writes east, north. - Repair lat/lon GeoJSON from ArcGIS WFSServer endpoints (WFS 1.1/2.0, EPSG:4326), which label it EPSG:4326 either way. Coordinates valid only one way round decide it; otherwise the feature type's WGS84BoundingBox from GetCapabilities does (lib/wfs-axis-order.ts). - On desktop, send WFS GetFeature through a new native fetch_url_response command, which ignores CORS and, unlike fetch_url_bytes, keeps the body of a non-2xx answer so the format fallback can read a 400 ExceptionReport. - Retry a GetFeature attempt once on a dropped connection or a gateway error; the PRG service resets a share of connections. - Keep a null geometry for attribute-only or empty GML features with honest typing instead of casting it away (review feedback), and fix the CRS limits in the docs. * Address review feedback - Cap the desktop's native WFS read at 128 MB (the WCS ceiling): the body is read into memory and crosses IPC as a JSON number array, so an unbounded response from a hostile or broken server could exhaust the app (Claude review, CodeRabbit). - Retry only network failures as classifyFetchFailure defines them, plus reqwest's generic send error and gateway 502/503/504, instead of any TypeError, so a parsing bug is not retried and disguised as a flaky connection (Claude review). |
||
|
Qiusheng Wu
|
d9d76518c8 |
feat(vector): adopt Add Vector Layer layers into native GeoJSON layers (#2716)
* feat(vector): adopt Add Vector Layer layers into native GeoJSON layers A file added through Add Data > Vector Layer got a much smaller Style panel than the same file dropped on the map: the control kept the features, so every section that needs layer.geojson (joins, virtual fields, attribute form, popup designer, editor tracking, diagrams, geometry generators, line decorations, blend mode) was hidden. Once the control finishes loading a GeoJSON-mode, table-ingested layer of up to 100,000 features, GeoLibre now reads its features into the store record, marks it maplibre-gl-vector-adopted, and removes the control's copy (map source and DuckDB table) in the same step. The swap stays out of the undo history and leaves the dirty flag alone. Container imports are grouped before their tables are adopted. The adopted record keeps its URL, file path and vectorState, so a saved project replays it through the control, which adopts it again, and a URL-backed layer refreshes the same way. A browser-picked file keeps its features in the project, like a drag-and-drop layer. Tiled, streamed, larger and KML-icon layers stay with the control. For those the Style panel now hides the controls the control ignores: fill patterns, stroke width units, proportional line widths, and the label zoom range, placement extras, expression and overrides. Fixes #2715 * Address review feedback - Remove the control's copy before marking the record adopted, so a throwing removeLayer leaves the layer control-drawn instead of adopted beside an orphaned control copy (CodeRabbit). - Treat an adopted layer awaiting its features as not yet drawable on the Mapbox and ArcGIS engines, which would otherwise compile a GeoJSON source without data or load its GeoParquet/GeoPackage URL as GeoJSON. Shared predicate isAdoptedVectorAwaitingFeatures lives in @geolibre/core (Claude review). - Prune declined adoptions per id alongside controlRenderState, so a later layer reusing a removed id gets a fresh look (Claude review). |
||
  Andrea BorrusoandClaude Opus 5.5
|
9668ab4608 |
feat(add-data): choose the CRS of a WMS layer in the desktop dialog (#2707)
* feat(add-data): choose the CRS of a WMS layer in the desktop dialog Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): address review on the WMS CRS selector Offer only CRSs the desktop tile protocol can reproject, keep saved CRS out of the web build, keep CRS:84's version and a desktop CRS on re-save, normalize restored codes, reset the CRS when the service query changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): check WMS CRS support per selected layers and on saved services Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): drop unoffered WMS CRS pick, keep version adoption on endpoint edit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): fall back to Web Mercator for a saved WMS CRS the version cannot request Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): note when no advertised WMS CRS can be reprojected on desktop Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): resolve the WMS CRS on submit, check reprojection for the web note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 * fix(add-data): check a saved WMS CRS pick for reprojection, note when it falls back Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3pudW5od717UbcEMXbTb1 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 Qiusheng Wuandgiswqs
|
275bdc8a2a |
feat(cad): render DXF/DWG coordinate Z values in 3D (#2567)
AutoCAD drawings routinely carry elevations (contours, 3D polylines, surveyed points). The DuckDB/GDAL read already preserved those Z values, but the layer was always handed to MapLibre's flat 2D style layers, so a contour set drawn at 0-250 m landed on the ground and the elevations were invisible. Add a "Render Z values in 3D" checkbox to the CAD (DXF/DWG) Add Data dialog, checked by default, which sets the layer's elevation3dEnabled style flag so it renders through GeoLibre's deck.gl Z-coordinate renderer the way a LandXML layer already does. The flag is inert for a drawing with no real Z (layer sync falls back to the 2D render), and the Style panel's Visualization radio switches a layer back to 2D at any time. Fixes #2559 Co-authored-by: giswqs <5016453+giswqs@users.noreply.github.com> |
||
|
Qiusheng Wu
|
03e650ba84 |
Add native LandXML import (#2492)
* Add native LandXML import * Add remote LandXML samples * Harden LandXML coordinate validation * Exercise LandXML fixture in parser tests * Validate LandXML URL source changes * Handle LandXML producer edge cases * Guard remote LandXML parsing * Handle complete LandXML curves * Respect LandXML CRS and unit semantics |
||
|
Daniel Hofbauer
|
9056a30d8f |
feat(services): add a runtime-configured deployment service library (#2457) (#2463)
* fix(services): let WMS and CSW forms accept same-origin relative endpoints Service forms only rejected relative URLs for the WMS and CSW kinds: both validated with an absolute-http(s) check before building requests, which blocked deployment-configured entries pointing at reverse-proxied services (/geoserver/wms, /catalog/csw). Introduce isServiceFormUrl (absolute http(s) or a same-origin reference), use it in both forms, resolve relative CSW endpoints against the document origin, and keep relative WFS/XYZ discovery out of the absolute-only Vite CORS proxy. * feat(services): add a runtime-configured deployment service library (#2457) Self-hosted deployments can now extend the built-in service library at runtime instead of maintaining a fork: a startup-provided catalog (VITE_GEOLIBRE_SERVICES) is merged as read-only entries that appear in the Browser and every Add Data service picker, are marked 'config' in all catalogs, are never stored in localStorage or exported, and cannot be edited or deleted. GEOLIBRE_BUILTIN_SERVICES=off (published by the entrypoint) hides the built-in starter presets while keeping configured and personal entries. * feat(docker): publish the deployment service catalog and built-in toggle The entrypoint reads an optional mounted GEOLIBRE_SERVICES_FILE on every container start, validates it strictly (nonblank id/name, known kind, nonempty primitive fields, unique ids; entry-indexed boot errors), and publishes it as VITE_GEOLIBRE_SERVICES in the runtime config, so a prebuilt image is repointed by restart alone. GEOLIBRE_BUILTIN_SERVICES=off hides the built-in starter presets; any other value fails the boot. The service worker stops precaching geolibre-runtime-config.js so deployment changes never get pinned to a build revision. * fix(services): address PR #2463 review findings - Reject protocol-relative URLs in isServiceFormUrl: they share only the scheme, not the origin, and would bypass the dev CORS proxy. - Reject relative WMS/CSW endpoints on the desktop (Tauri) build with the translated URL message; the native HTTP path requires an absolute http(s) endpoint. Also guard the Browser apply path so it no longer surfaces the raw 'Invalid WMS tile URL.' throw. - Resolve relative CSW endpoints against window.location.origin instead of the full href, so route-relative entries are path-independent. - Reject integers beyond JavaScript's safe range (2^53-1) in the entrypoint catalog validation: they would become Infinity or round during JSON.parse. - Fix the Thai config label (Bengali codepoints) and translate the Dutch label to geconfigureerd. - Docs: clarify that startup validates structure only (the forms validate kind-specific fields when applied), list every kind's field shapes, mention the safe-integer bound, and fix the broken getting-started link from docs/user-guide. * fix(services): reject scheme-only URLs and precise safe-integer wording Apply the second CodeRabbit pass on #2463: - isServiceFormUrl now parses absolute http(s) candidates, so scheme-only values like 'https://' fail at the form with the translated error instead of throwing from new URL() in the request builders (web and Tauri alike). - The entrypoint's field error and docs now state that finite numbers are allowed and only integers must be within the safe-integer range (2^53-1). |
||
|
Qiusheng Wu
|
f093025735 |
Add WCS 1.0 coverage discovery and numerical raster subsets (#2460)
* Add WCS 1.0 coverage discovery and GeoTIFF subset loading * Address review feedback - Enforce the WCS native response limit while streaming and stop waiting promptly on cancellation. - Clarify the empty coverage message across all locales. - Consolidate WCS documentation into a supported data formats reference with navigation and guide links. * Address review feedback - Prefill the WCS endpoint from a deep link: `AddDataDialog` now passes `initialUrl` to `WcsSource`, which seeds its endpoint field from it, and `wcs` joins the `?add=` service kinds so the prop has a producer. Braces are left encoded, as for every other KVP service (CodeRabbit, AddDataDialog.tsx). - Document why `xmlDocument` accepts a 1.0.0-rooted document that omits the `version` attribute: the root element name is the version gate — WCS 1.1 and 2.x answer with `Capabilities`/`CoverageDescriptions` and are rejected a line earlier (Claude review, wcs.ts). * Address CodeRabbit review feedback - Bound WCS metadata responses: `fetchCapabilitiesText` takes an optional byte ceiling, enforced by the native command's existing `maxBytes` and by a new `readLimitedBody` on the browser path, which counts the stream instead of buffering it whole. `discoverWcs`/`describeWcs` cap listings and coverage descriptions at 32 MB and surface a translated `metadata` error; the coverage download now shares the same reader. Other OGC callers pass no limit and are unchanged (wcs-fetch.ts). - Drop a coverage whose decode outlived a cancelled request, so closing the dialog mid-decode no longer leaves the layer it cancelled behind (WcsSource.tsx). - Fix the Hindi coverage-extent label: "कवरेज विस्तार का उपयोग करें" (hi.json). * Address Claude review feedback - Route non-WCS errors through the shared `serviceRequestErrorMessage`, so a network, CORS, or timeout failure shows the same localized hint as the WMS/WFS panels instead of a raw `TypeError: Failed to fetch`. - Give the coverage picker its own `addData.wcs.coverage` label in all 19 locales rather than borrowing `printLayout.atlas.coverage`, whose translations were authored for the Print Layout atlas (Vietnamese rendered it as "Bảo hiểm", insurance). |
||
|
Qiusheng Wu
|
95c0971bc5 |
docs: add a Web Services page to the user guide (#2251)
* docs: add a Web Services page covering all 17 submenu plugins * docs: add screenshots to the Web Services page * docs: point Web Services screenshots at the published asset URLs * docs: drop the USGS NLDI nav entry The page stays reachable from the Web Services and Plugins pages, so it does not need its own User Guide nav slot. * Address CodeRabbit review feedback - Reword "Each one is documented on its own page" to say all seventeen browsers live on the single Web Services page. - Note that Web Services panels dock rather than float, so the blanket "you can also set their on-map position" sentence does not apply to them (Vantor Open Data and Planet Open Data included). * Address CodeRabbit review feedback Scope the "activated from the Plugins menu" sentence to the three tables it actually covers. Planetary Computer and Earth Engine are reached from the Processing menu and STAC from Add Data, so the blanket "all of these" was wrong for the top of the page. |
||
  
|
353d29021a |
feat(geoparquet): parse the full geo metadata block, native logical-type CRS, and lon/lat tables (#2156)
* feat(geoparquet): parse the full geo metadata block and native logical-type CRS GeoLibre read a GeoParquet's `geo` metadata for its CRS and nothing else: `version`, `primary_column`, `encoding`, `geometry_types`, `bbox`, `covering` and `edges` were all dropped, and a Parquet 2.0 file recording its CRS only on the geometry column's GEOMETRY/GEOGRAPHY logical type had no CRS at all. Add `geoparquet-metadata.ts`, a DuckDB-free module that parses the whole block and the logical type, and reduce `geoparquet-crs.ts` to the SQL-building layer over it (every exported signature and every existing test is unchanged). The parsing rules are ported from GeoPQ Workbench, which has been reading these files in the field: - A `bbox` is ordered minima-first, so a 6-element 3D box is [xmin, ymin, zmin, xmax, ymax, zmax]. Taking its first four elements — the obvious reading — yields a box whose ymax is the minimum elevation. - A `covering.bbox` is accepted only in the one layout a reader can act on: four ["<struct>", "<field>"] paths under a single root. - An absent `crs` member (the OGC:CRS84 default) and an explicit `"crs": null` are now distinguished. Both skip reprojection, but only the second may not be labelled WGS84, so `readGeoParquetGeoMetadata` exposes which one it was. - A PROJJSON `id.code` may be a number or a string; OGC:CRS84/84, CRS83 and CRS27 map onto EPSG:4326/4269/4267, which PROJ resolves far more reliably. - Id-less geographic PROJJSON renders as lon/lat without claiming a datum, rather than handing PROJ a document it may not resolve. Geometry detection follows the evidence, best first: the column the `geo` block declares primary, then a native GEOMETRY column, then the WKB name guesses, and finally — new, and only on the Parquet load path — a lon/lat pair of float columns synthesized into points with ST_Point, so a plain point table published as lon/lat columns opens instead of failing with "no geometry column". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * test(geoparquet): add shared GeoParquet fixtures with a regenerate script No test anywhere in the repo read a real Parquet footer, so every GeoParquet rule was asserted against hand-written JSON that nothing guaranteed a writer would ever emit. Add seven small files (3-19 kB each) under tests/fixtures/geoparquet, one per variant the app has to open: 1.0 WKB, 1.1 with a declared covering bbox struct in EPSG:26986, 2.0 with the native GEOMETRY logical type, an explicit `"crs": null`, a 6-element 3D bbox, a plain Parquet whose only geometry is a WKB blob named `geom`, and a table of lon/lat columns with no geometry at all. `regenerate.sh` rebuilds them from the DuckDB CLI plus a pyarrow footer patch for the `geo` blocks DuckDB cannot write, and verifies each one with `parquet_schema`. `expectations.json` records the expected outcome per file — version label, CRS kind and EPSG, encoding, covering, 2D bbox, detection route — and is consumed by both suites, so the TypeScript and Python detection rules cannot drift apart silently. Each file's `geo` document is extracted at generation time into geo-metadata/, keeping the TypeScript tests free of any Parquet reader. The sidecar test pins its own detection, including the lon/lat gap: the browser loader synthesizes points from such a pair, the sidecar asks the caller to name the columns, and that is now a recorded decision rather than a surprise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * docs(geoparquet): describe the variants that open and how an undefined CRS reads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * style: auto-format (ruff + oxfmt) [pre-commit.ci] * test(geoparquet): declare the 3D fixture's bbox as its true extent The hand-written 6-element bbox stopped at ymax 42.48 while the generator writes latitudes up to 42.48235294117647, so the declared bounds excluded valid rows. Declare the actual extent, as the other fixtures do. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * fix(geoparquet): synthesize points only when no geometry column can exist `detectGeometryColumn` fell back to a lon/lat pair whenever no geometry column was recognised, so a WKB blob under an unrecognised name plus an unrelated numeric `x`/`y` pair produced a layer of bogus points over the real data. Require the description to hold no GEOMETRY-typed column and no binary column at all before the coordinate pair is promoted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * feat(geoparquet): give the native DuckDB loader the same metadata rules On desktop the Tauri loader tries the native (Rust) DuckDB path first and only falls back to DuckDB-WASM on error, so its detector has to agree with `duckdb-geometry.ts` / `geoparquet-metadata.ts` or the same file opens differently on desktop than in the browser. It did not: it never read `primary_column`, resolved the CRS against a fixed `columns` key rather than the column actually detected, had no Parquet 2.0 logical-type CRS fallback, and could not synthesize points from lon/lat columns. The `geo` document is now read once per load and reused for both detection and the CRS. Ported from the TypeScript, rule for rule: the absent / explicit-null CRS distinction, the PROJJSON `id` (numeric or string code, OGC:CRS84/84/CRS83/CRS27 mapped onto their EPSG codes), id-less PROJJSON handed to `ST_Transform` whole, the pre-1.0 raw string, and — for a 2.0 file with no `geo` block at all — the `parquet_schema()` logical-type CRS in each spelling in circulation. A second geometry column is also dropped from the properties: the DuckDB client has no reader for the spatial extension's type and selecting one fails the whole query, which is exactly the shape of the multi-geometry file a declared `primary_column` exists for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * refactor(geoparquet): read the geo block through a single entry point `geoParquetCrsEpsg` and `readGeoParquetGeoMetadata` were both dead as far as app code went. The loader now calls `readGeoParquetGeoMetadata`, which is what it was written for, and the `geoParquetSourceCrs` wrapper that stood in front of it goes with it rather than becoming the unused export in its place. `geoParquetCrsEpsg` was only ever an assertion helper, so it moves into the test that used it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * docs(geoparquet): say what the shared fixtures actually cover The module docstring claimed the detection rules live in "three places" without saying that Rust reads these fixtures too. It now names each implementation and points at the cargo tests that read the same files, and records that the sidecar deliberately has no lon/lat synthesis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * fix(geoparquet): fall back to the logical-type CRS when the geo key does not parse A geo key that is not JSON took the metadata branch and returned no CRS, so a Parquet 2.0 file with a damaged geo key and a projected CRS on its GEOMETRY logical type loaded in raw units under the WASM loader while the native loader, which parses first, reprojected it. Gate on the parsed document, as the native loader does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * test(geoparquet): build the native loader's inputs in-test `native_loader_synthesizes_points_from_lon_lat_columns` and `native_loader_skips_reprojection_for_an_explicit_null_crs` were the only cargo tests reading the committed `tests/fixtures/geoparquet` files. They now write their own inputs with the bundled DuckDB, the way `create_real_geoparquet` and the declared-primary-column test already do: a lon/lat table straight from `range()`, and the undefined-CRS case as a WKB blob plus a `KV_METADATA {geo: ...}` block carrying `"crs": null` (DuckDB emits no `geo` key of its own for a blob column, so ours is the only one). Both clean up after themselves like their neighbours. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * test(geoparquet): generate test inputs in place of committed fixtures The seven Parquet files, their extracted `geo` sidecars, `expectations.json` and `regenerate.sh` under `tests/fixtures/geoparquet` are gone; nothing in the repo needs a committed binary to exercise GeoParquet reading any more. - `tests/geoparquet-metadata.test.ts` carries the same seven variants as an in-file table: each case holds the `geo` document a writer emits plus the DESCRIBE rows and `parquet_schema()` logical types DuckDB 1.5.4 reports for it. Every assertion is unchanged, and the suite still needs no Parquet reader. - `test_geoparquet_fixtures.py` becomes `test_geoparquet_detection.py` and writes each input into `tmp_path` with duckdb + spatial at test time, so the expectations stay honest about what a real writer produces. A new smoke test runs the sidecar's script straight over the SQL Workspace's own sample dataset (`data.source.coop/.../countries.parquet`, ~200 kB) to cover the HTTP `read_parquet` path; it skips when a two-second connect to `data.source.coop:443` fails. Dropped with the fixtures: the sidecar's 3D-bbox and `crs: null` cases. DuckDB cannot write those `geo` blocks and pyarrow is not in the backend's `test` extra, and the sidecar only detects the geometry column — it never reads `crs` or `bbox` — so neither case tested anything there. Both are still pinned in the TypeScript table and the cargo tests, which do read them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA * test(geoparquet): probe the sample dataset's host inside the test, not at import The skipif marker evaluated the socket probe when pytest imported the module, so collecting the file opened a connection and could block for two seconds even when the remote test was deselected. A fixture defers the probe to the one test that reads the sample. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LRx22ZHWRiPHrxf6F72sA --------- Co-authored-by: Guillaume SUEUR <gsueur@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
 Qiusheng Wuandgithub-actions[bot]
|
5ecb05760e |
docs: correct the user guide against the running app and add screenshots (#2112)
* docs: correct the user guide against the running app and add screenshots Several pages had drifted from the shipped UI: the toolbar was documented as seven menus with Edit and View missing entirely, Add Data omitted ten entries, the plugin and style-type lists were from an older release, and the terrain tutorial claimed raster tools need the desktop app when eight of them have a browser engine. Every claim here was rechecked by driving the app, and the Browser panel, Model Builder, Dashboard, and the attribute table's own tools are now documented rather than only mentioned in the feature list. * fix(mkdocs): update GeoLibre Web link in documentation * Address Claude review feedback - Correct the `C` shortcut row: it toggles the review-comment tool and the pin lands where you click next, not at the map center. Also mention the shortcut in the Review comments section, which only documented the panel. - Give Managing Layers its own Layers-panel screenshot instead of reusing the full-workspace shot from Interface Overview, which did not match the alt text. * Address CodeRabbit review feedback - Name both projection-aware engines in the buffer tip. Python (Pyodide) runs the same GeoPandas path as the sidecar and is the one available on the web build, so pointing only at the sidecar sends web users nowhere. - Stop claiming every on-map-control plugin has a positioning submenu. Flight Simulator and SamGeo toggle directly, and Web Services and DGGS open a list of sub-plugins instead. * Update mkdocs.yml Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Qiusheng Wu
|
9dbd684198 |
fix: read the source CRS of a GeoParquet from its geo metadata (#2094)
* fix: reproject GeoParquet from the CRS its geo metadata declares
Every other vector format goes through GDAL's ST_Read, whose ST_Read_Meta
reports the layer CRS, so the loader reprojects it to WGS84. GeoParquet is read
with read_parquet instead and DuckDB surfaces nothing about its CRS in the scan,
so readSourceCrs returned null for it outright: a file stored in a projected CRS
loaded as raw eastings/northings and the map refused to draw it ("Coordinates in
<layer> are not longitude/latitude"), reported for EPSG:2100 (GGRS87 / Greek
Grid) in issue #2086.
The CRS is in the file. The GeoParquet specification writes it into the Parquet
file-level key/value metadata under the key `geo`, which DuckDB exposes via
parquet_kv_metadata. The loader now reads that document and resolves the primary
geometry column's `crs`, preferring the PROJJSON `id` (EPSG:2100), falling back
to the PROJJSON itself for a CRS with no authority code, and to the raw string
for the pre-1.0 drafts that wrote WKT2. Reprojection is still skipped for every
"already lon/lat" case the spec spells out: an absent `crs` member (the default
is OGC:CRS84), an explicit "crs": null, and a WGS84/CRS84 identity.
The parsing lives in its own leaf module so it is unit-tested without pulling
duckdb-wasm into the coverage denominator.
Also adds GGRS87 (EPSG:4121) to the Processing CRS picker, whose projected list
already carried the Greek Grid but not the geographic system it is based on.
* fix: resolve the GeoParquet CRS for the column being read
A GeoParquet may carry several geometry columns in different CRSs, and the
loader reads whichever column it detected rather than necessarily the one the
document calls primary. Resolving `primary_column`'s CRS in that case would
transform the loaded geometry with another column's projection and land the
layer somewhere else entirely, so the detected column name is now passed down
and looked up first, falling back to `primary_column` and then to the first
column described.
* chore: bump maplibre-gl-vector to 0.10.13
Picks up opengeos/maplibre-gl-vector#69, which resolves a GeoParquet source CRS
from the file's own `geo` metadata. Add Data > GeoParquet Layer routes to that
package's panel, so before the bump a GeoParquet in a projected CRS loaded there
sat at "Converting ... to GeoJSON..." forever unless the Source CRS override was
filled in by hand.
|
||
|
Qiusheng Wu
|
5a928c02de |
Add ImageServer raster function browser (#2043)
* feat(arcgis): browse ImageServer raster functions * fix(arcgis): preserve custom rendering rules * fix(arcgis): avoid raster function option collisions |
||
|
58d08f4ee7 |
feat: Encoded Polyline Encoder/Decoder Implementation (#2030)
* feat: Encoded Polyline Encoder/Decoder Implementation Implemented a comprehensive Google Encoded Polyline codec supporting standard precision 5 (Google, OSRM) and precision 6 (Valhalla, Mapbox), unescaping of backslashes, batch parsing with custom delimiters, interactive preview and coordinate inspector in Add Data UI, processing toolbox tools, layer export, and full Python package support with zero external dependencies. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix: Sequence was used in the type annotation of add_polyline but was never imported Sequence was used in the type annotation of add_polyline() at line 1203, but was never imported. * fix(polyline): resolve review findings across codec, export, and processing tools for Encoding Polyline #2029 - PolylineSource: use translated previewInfo string and reset delimiters on sample select - tauri-io: deduplicate polyline decoding and validate coordinate ranges for precision detection - vector-export / UI: thread precision option and add distinct precision 5 & 6 export menu actions - core & python: prevent unsafe conversion of literal \n, \r, \t in unescapePolyline - processing: validate polyline ASCII character range and handle MultiLineString roundtrips symmetrically - python: import Sequence from collections.abc and align delta rounding to Math.round() behavior * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(polyline): localize UI labels and harden coordinate validation in processing and Python API #2029 - LayerPanel & AttributeTable: localize Encoded Polyline export options via dynamic precision catalog key - PolylineSource: localize sample route labels and metric distance units via t() - vector-tools: add bounds validation ([-180, 180], [-90, 90]) for decoded coordinates in decodePolylineTool - tests: add null assertion on decoded geometry and test for out-of-bounds polyline input - python/polyline: validate coordinate bounds in encode_polyline, check character byte range in decode_polyline, and default unescape to False * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(polyline): enhance decoder validation, geometry export restrictions, and UI formatting #2029 - packages/core: export `decodePolylineDetailed` to indicate whether polylines were fully consumed without truncation. - packages/processing: update `decodePolylineTool` to skip features with incomplete coordinate pairs using `decodePolylineDetailed`. - apps/geolibre-desktop: - Format distance preview using locale-invariant SI units (`m` / `km`). - Sanitize `baseProperties.source` to file basename instead of full local path. - Strict line-compatibility check in `layerSupportsPolylineExport` to reject mixed-geometry layers. - python: validate cumulative lat/lon bounds `[-90, 90]` and `[-180, 180]` in `decode_polyline`. - tests: add regression tests for truncated coordinate decoding, mixed geometry exports, and Python coordinate bounds. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(polyline): reject overlong varint shifts and harden geometry export restrictions #2029 - packages/core: guard polyline varint decoding loops against shifts exceeding 30 bits or 32-bit payload overflows, returning incomplete status immediately. - apps/geolibre-desktop: ensure layerSupportsPolylineExport strictly requires all feature geometries to be line-compatible. - tests: add regression tests for the overlong "_______?" varint chunk in decodePolyline and decodePolylineTool. - python: add tests covering coordinate bounds validation on latitude and longitude axes. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(polyline): correct sample escaped data and harden coordinate validation #2029 - apps/geolibre-desktop: - Update `sampleEscaped` sample polyline in `PolylineSource.tsx` with a valid string containing a properly escapable backslash varint sequence. - Harden `hasValidPolylineCoordinates` in `tauri-io.ts` to reject empty coordinate lists and enforce non-empty total point counts. - Document precision 5 vs 6 auto-detection heuristics and bounds handling during dropped-file ingestion. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(polyline): enforce complete decoding in batch decoder and clean up imports #2029 - packages/core: require `complete === true` from `decodePolylineDetailed` in `batchDecodePolylines` before accepting features. - apps/geolibre-desktop: remove unused `Info` and `MapPin` imports from `PolylineSource.tsx`. - tests: add test verifying `batchDecodePolylines` skips lines with invalid characters or truncated varints. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * Address review feedback - Fix the "Escaped backslashes (JSON string format)" sample, which failed to decode at all ("No valid polyline coordinates could be decoded"). The old value was not a real polyline: it carried one stray backslash that truncated a varint, and even ignoring that it decoded to coordinates off the planet (lat 130, lon 774). Replaced with a San Francisco -> Monterey route whose encoding genuinely contains a backslash, written doubled the way a JSON payload carries it, so the dialog's unescape step is what makes it decode. - Move SAMPLE_POLYLINES out of PolylineSource.tsx into a leaf module, lib/polyline-samples.ts, so the samples can be decoded in a test rather than only in the UI, and add tests/polyline-samples.test.ts covering every sample and the escaped one's with/without-unescape behaviour. - Document in parsePolylineFileLayers that the precision 5 -> 6 bounds check cannot resolve the ambiguous cases: the two decodes differ by exactly 10x, so in-bounds at precision 5 implies in-bounds at precision 6, and precision-6 data near the prime meridian/equator still imports 10x too large. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: giswqs <giswqs@gmail.com> |
||
|
Qiusheng Wu
|
8cc9f0f158 |
docs: link the published Chrome Web Store listing (#1989)
* docs: link the published Chrome Web Store listing The Open data in GeoLibre extension is live on the Chrome Web Store, so add a store badge to the README and make the store the primary install path in the docs, keeping the unpacked release as the manual alternative. * docs: add the Chrome Web Store badge to Getting Started |
||
|
Qiusheng Wuandpre-commit-ci[bot]
|
5ad512332e |
Add Chrome extension for opening webpage datasets (#1932)
* feat: add GeoLibre Chrome data opener * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix: address Chrome extension review feedback * ci: attach Chrome extension to releases * docs: install Chrome extension from release * refactor: rename Chrome extension package * feat: link Chrome extension to GeoLibre * style: auto-format (ruff + oxfmt) [pre-commit.ci] * ci: disable caching in release builds --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Qiusheng Wu
|
e5f8f0950f |
docs(processing): explain the two toolboxes and fix stale menu paths (#1929)
* docs(processing): explain the two toolboxes and fix stale menu paths
The Processing menu carries two independent tool collections whose category
names collide: the Whitebox toolbox catalog (Conversion, Hydrology, LiDAR,
Network, Projection, Raster, Remote Sensing, Terrain, Vector as direct children
of the menu) and GeoLibre's own dialogs under the GeoLibre Toolbox submenu.
Nothing in the docs said so, so 'Processing / Vector / GeoLibre (WASM)' and
'Processing / GeoLibre Toolbox / Vector' read as duplicates.
- New 'Two toolboxes in one menu' section on the Processing Tools page: the
menu tree as rendered, a side-by-side table of what each toolbox is and
where it runs, and the specific Vector-vs-Vector pair spelled out. Also
notes that the dialog's Source filter ('GeoLibre tools' / 'Whitebox tools')
splits the Whitebox catalog only, which is a third thing again.
- Reorganized the page under 'GeoLibre Toolbox' and 'Whitebox Toolbox'
headings and documented the families that had no entry at all: Network,
Spatial Statistics, DGGS, Geocode Addresses, Batch & Models, Object
Detection, Segment Everything, plus the Raster Analysis group and the
Vector Movement and Data quality groups.
- Corrected the Conversion section: every conversion tool now has a
client-side engine, so the per-tool 'Sidecar' engine column was wrong;
Raster to PMTiles was missing entirely.
- Updated the menu paths that moved under GeoLibre Toolbox across the docs,
README, and the sidecar README, and repointed the README's
#geoprocessing-toolbox anchor at #whitebox-toolbox.
- Added a Troubleshooting entry for users who land there after seeing the
same category name twice.
* docs(processing): scope the conversion allowlist claim, note colliding tool names
Two points from review.
The conversion sidecar's path allowlist is opt-in, not unconditional:
GEOLIBRE_CONVERSION_ROOTS is unset on desktop (app/conversion.py: 'Unset
means no restriction (the default for the desktop app)') and set by the
Docker image, whose sidecar is reachable same-origin through the nginx
proxy. The page inherited a flat 'hardened with a path allowlist' claim
that read as always-on; it now says which deployment sets it and why.
Reviewing a (mistaken) claim that the tutorials' Conversion paths pointed
at the wrong toolbox turned up a sharper case of this PR's own subject:
a few individual tool *names* collide across the toolboxes, not just
category names. The WASM catalog carries vector_to_pmtiles / write_pmtiles
displayed as 'Vector to PMTiles' and 'Raster to PMTiles', which are also
the names of two ConversionDialog tools, so both menus show a 'Vector to
PMTiles' leaf that opens a different dialog. Called out in 'The two paths
that look alike', with a note that these docs mean the GeoLibre Toolbox
one. The existing paths are correct and unchanged.
* docs(processing): correct the separator reference and the result-is-a-layer claim
Two minor review points, both right.
'Everything above the first separator' put AI Assistant inside the
Whitebox block; the Whitebox section is the block between the first and
second separators.
'the result is added to the map styled and ready' held for the analysis
dialogs but not for Conversion, which writes a file. Split the two.
|
||
|
Qiusheng Wu
|
30126bab6e |
feat(arcgis): add MapServer and ImageServer support to Add Data (#1751)
* feat(arcgis): add MapServer and ImageServer support to Add Data
Add Data > ArcGIS Layer only offered FeatureServer and VectorTileServer,
so the two service types that serve rendered images had no way in.
Both now load as ordinary raster layers, which is what makes the whole
raster surface work on them (opacity, the Style panel's brightness /
contrast / saturation controls, reordering, and project save and reload)
without a dedicated handler anywhere else in the app. Tiles come from the
service's own fused cache when it was built on the standard Web Mercator
scheme, and from `/export` (MapServer) or `/exportImage` (ImageServer) as
a `{bbox-epsg-3857}` template otherwise; a cache in another projection or
with a custom LOD table would render misaligned, so it falls back to the
dynamic endpoint rather than drawing something wrong.
Two optional fields shape what the service draws: MapServer sublayer ids
(`layers=show:0,2,5`, also read off a pasted `.../MapServer/3` URL) and an
ImageServer rendering rule. Either forces the dynamic path, because a
cache was rendered before the choice existed and cannot honor it.
Fixes #1746
* Address CodeRabbit review feedback
- Read `sublayers` only for a map service and `renderingRule` only for an
image service. The Add Data form keeps both field values when the layer
type is switched, so either could arrive on the wrong service type: an
unparseable leftover rendering rule blocked a MapServer submission, and a
parseable one silently cost it its tile cache.
- Validate the cached tile origin's Y coordinate as well as its X. A cache
anchored at the correct left edge but a different top edge passed the check
and rendered horizontally aligned and vertically shifted, which reads as
imagery in the wrong place rather than an obvious break.
- Assert the egress redaction that keeps an access token out of a shared
project, instead of trusting `metadata.hasAccessToken` alone.
- Match each catalog's own grammatical form in the Arabic, German, and Russian
Add Data descriptions (verbal noun / third person / infinitive), which the
rewrite for the four service types had changed.
|
||
|
Qiusheng Wu
|
f879b675ed |
fix(vector): stream large local vector files through DuckDB (#1716)
* fix(vector): guard large local vector loads before reading the file A local vector file had no size guard of any kind. Two cases froze the app with no actionable error: - A GeoJSON at or above V8's maximum string length (~537 MB) cannot be read as text at all — `readTextFile`/`File.text()` throw `RangeError: Invalid string length` before `JSON.parse` runs. The loaders' `catch` swallowed it and silently re-read the whole file through DuckDB, so the user waited through the failed attempt first. - A large zipped shapefile went to shpjs, whose `parseShp` is fully synchronous and applies the `.prj` proj4 transform per coordinate, wedging the main thread with no progress, no cancel, and no feature-count guard (that guard only runs once DuckDB has already opened the source). Add a byte-size preflight that runs on filesystem metadata (`stat`) or `File.size`, before any read: - `LARGE_VECTOR_SIZE_WARN_BYTES` (200 MB) prompts via the new `onLargeFile` option, wired alongside `onLargeDataset` at every interactive entry point (drag-drop paths and files, the desktop picker, KML import). - `MAX_TEXT_VECTOR_BYTES` diverts an oversized GeoJSON/KML/GPX/CSV straight to the DuckDB reader instead of attempting the doomed text parse. - `MAX_SHPJS_SHP_BYTES` (64 MB, measured on the uncompressed `.shp`) sends a large shapefile to DuckDB, which reads off the main thread and reports a feature count first, so the existing large-dataset guard can fire. Also document the practical ceilings and the convert-first workaround in the Adding Data guide. * docs(vector): record the measured cost of the large-vector guards Verified against a real 197 MB / 170k-polygon wetlands shapefile and an 873 MB GeoJSON in the browser. Two claims in the original comments did not survive measurement: - The text-limit diversion saves ~2s, not a full wasted read: File.text() rejects on the known size rather than after reading the file. The value is the explicit route and the log line, not the time saved. - The shpjs bypass trades total time for responsiveness rather than being a pure win: worst main-thread stall 8.9s -> 1.6s (projected .prj) and 4.7s -> 1.9s (WGS84 .prj), while overall load time rises by 3-4s. Record both numbers next to the constants so the trade-off is not rediscovered later. * feat(vector): stream local files over 100 MB through DuckDB Replace the size-confirmation prompt with automatic routing, and lower both thresholds to the configured defaults: - DUCKDB_VECTOR_ROUTE_BYTES (100 MB): at or above this, a local vector file skips the in-memory JavaScript readers (JSON.parse for GeoJSON, shpjs for shapefiles, the KML/GPX/CSV text readers) and streams through DuckDB instead — off the main thread, no prompt, just a console note. For a zipped shapefile the threshold is measured on the uncompressed .shp, since shapefiles compress heavily. - DUCKDB_VECTOR_FEATURE_WARN_COUNT: 500,000 -> 100,000. This stays a prompt, because it is about materializing features in memory, which file size does not predict for compressed formats. One threshold now covers every format, replacing the separate 64 MB shapefile rule. MAX_TEXT_VECTOR_BYTES is gone: routing at 100 MB is far below V8's 536,870,888-byte string cap, so the RangeError it guarded is unreachable. Removes onLargeFile, confirmLargeVectorFile, LargeVectorFile, and the largeVectorFileDesc string from all 17 locales. * fix(vector): unify the Add Vector Layer threshold, warn on mislabelled CRS, and stop autosave crashing on large projects Three fixes to the large-vector handling, all found while testing a real 170k-polygon wetlands dataset. 1. Unify the Add Vector Layer threshold. The panel is backed by maplibre-gl-vector, whose autoThreshold defaulted to 25 MB / 50k features, while the drag-and-drop loaders switched to DuckDB at 100 MB / 100k. The same file therefore behaved differently depending on how it was added, and no single limit could be documented. Both thresholds now live in @geolibre/core and are passed to the control, so the two entry points agree. 2. Warn when a layer's coordinates cannot be WGS84. detectNonGeographicCoordinates samples a collection and reports coordinates outside +/-180 / +/-90 — the case where a file declares CRS84 or GCS_WGS_1984 but holds projected easting/northing. GeoLibre honours the declared CRS, so such a layer loads cleanly, lists in the Layers panel, and renders nowhere with no error at all. It now says so, in its own toast: the drop handler sets a success message after the layers are added, which would clobber a shared one, and both messages are true at once. 3. Stop autosave crashing. serializeProject runs synchronously outside the promise chain in useProjectHistory, so a project embedding a large vector layer threw an unhandled `RangeError: Invalid string length` (V8's 536,870,888-byte string cap) on every autosave tick. Autosave is best-effort and must degrade to "no crash recovery", not to a crash. Verified against the mislabelled CT_Wetlands data: the warning fires with the real out-of-range values and stays quiet on the reprojected copy. * Address review feedback - Keep delimited text on the JS parser at any size. loadDuckDbVectorFile has no lon/lat column detection (that lives only in the GeoParquet conversion path), so routing a plain lon/lat CSV over 100 MB to DuckDB failed with "DuckDB did not find a geometry column in this file" for files that loaded before. Reported on both loadTauriVectorFile and loadBrowserVectorFile. - Do not abort a drop batch on an oversized GPX/KML. Those branches read the whole document as text to extract ground overlays and models before the guarded loader runs; that read now decodes from bytes past the string cap, and a failure yields no overlays instead of throwing out the whole batch. - Only log the DuckDB route for extensions the flag actually gates. zip/kmz unpack first and decide from the uncompressed .shp, so the container's own size made the message misleading near the threshold. - Visit GeometryCollection members in detectNonGeographicCoordinates. Their coordinates sit under geometries[], so such features were skipped entirely and an all-GeometryCollection file passed the check silently. - Clear the CRS warning when a new drop starts, so it cannot linger over an unrelated drop the way dropMessage/dropError cannot. - Correct the docs: PMTiles is the tiled format; GeoParquet is columnar and compact but not tiled. * Address CodeRabbit review feedback - Separate snapshot construction from serialization in the autosave path. The single try block also covered buildProjectSnapshot, so a construction failure was misreported as "the project is too large". - Make the mislabelled-CRS warning count-neutral in all 17 locales. The importer aggregates layer names into one message, so wording that agreed with a singular subject was wrong for a multi-file drop. - Route an oversized GPX through the guarded loader instead of decoding the whole document into one string. The DuckDB reader returns a single merged layer rather than the waypoint/track/route split, which is the accepted trade at that size. - Clear the CRS warning when a browser drop starts. It was previously cleared only on the 4s status timeout, which both missed the browser handler and wrongly auto-dismissed a warning that has its own Close button. - Make the sampling-limit test observable by placing an out-of-range feature past the limit; the all-valid collection passed even if sampleLimit were ignored. * Address Claude review feedback - Gate the maxAbsX/maxAbsY update on Number.isFinite, matching the finiteness check already used for the offending determination. A single Infinity would otherwise be reported as the offending magnitude and hide the real value. - Clear the CRS warning in the KML import handler too. Its catch sets dropError without reaching addImportedVectorLayers, so a previous file's banner could sit beside the new error. * chore(deps): bump maplibre-gl-vector to 0.10.8 Picks up opengeos/maplibre-gl-vector#59, which fixes two bugs the Add Vector Layer panel hit on a large, custom-CRS GeoPackage: - A layer whose GeoJSON document exceeded the maximum JavaScript string length failed to ingest with "Invalid string length". It is now encoded feature by feature and ingested in batches. - A layer whose gpkg_spatial_ref_sys row is not an EPSG row (organization "CUSTOM", carrying a WKT definition) was never reprojected, so it rendered off the map with "Invalid LngLat latitude value" and no explanation. Per the bump checklist in CLAUDE.md, MAX_VECTOR_BYTES in packages/plugins/src/plugins/remote-file-formats.ts was re-checked against MAX_REMOTE_FILE_BYTES in the package's src/lib/utils/remote.ts: both are still 2 ** 31 - 1, so the mirror needs no change. Verified in the built web app: a GeoPackage with a CUSTOM-organization Albers SRS now reprojects and renders over Connecticut with zero diagnostics. * Address Claude review feedback - Stop routing large GPX through DuckDB. `loadDuckDbVector` passes no `layer` argument, so `ST_Read` reads only a GPX's first OGR layer (usually `waypoints`) and silently discarded its tracks and routes — data loss, not the "single merged layer" the comment claimed. GPX now always uses `parseGpxTextLayers`, matching the decision already made for delimited text. - Drop `readVectorFileText`. `TextDecoder.decode()` over the whole buffer builds one JS string exactly as `File.text()` does, so it hit the same RangeError one line later; the comment claimed an avoidance that never existed. `readVectorFileTextOrEmpty` now calls `file.text()` directly and keeps the catch, which is what actually made an oversized KML survivable. - Report a non-RangeError serialization failure as an error rather than filing it under "the project is too large", which would have hidden that class of bug. |
||
|
Qiusheng Wu
|
1483fcef14 |
docs: fix stale content, cover shipped features, drop hardcoded versions (#1634)
* docs: fix stale content, cover shipped features, drop hardcoded versions Documentation review pass across docs/. Structure - Add collaboration.md to the mkdocs nav; it was on disk and linked from index.md and features.md but absent from the sidebar. - Break up the run-on bullets in features.md into nested lists. The longest line drops from 2,189 to 470 characters, with no facts removed. - Replace index.md's 5,584-character "Project status" paragraph with a scannable list. Accuracy - projects.md documented "Project -> Print...", which does not exist; the menu item is "Print Layout...". Rewritten to match the actual composer. - Import QGIS Project sits under a "Import" submenu, not the top level. Coverage: features shipped with no documentation - Autosave, crash recovery, and project history (projects.md) - QGIS project import (projects.md) - Nested layer groups and multi-select moves (layers.md) - Anchored review comments (map-controls.md) - Elements panel for annotations (map-controls.md) - NMEA GPS/GNSS receiver support (map-controls.md) - ArcGIS Hub, Socrata, and CKAN browsers (adding-data.md) - KML Super-Overlays (adding-data.md) - Persisted layer connection records (layers.md) - Dashboard selector widget (features.md) map-controls.md was missing 11 of the Controls menu's tools and projects.md was missing 6 Project menu entries; both now cover their full menus, with a note that the UI profile can hide entries. Version numbers - Drop the pinned "GeoLibre 2.4"/"v2.4"/"Since 2.4" claims from index.md and features.md, and the per-version highlights table that duplicated roadmap.md, so a release does not leave three places to update. - architecture.md said "The v1.0 Add Data surface" and project-format.md had a "v1.0 status" column while describing current behavior. - Version numbers that are historical facts are left pinned: the roadmap's release history, the macOS signing threshold (v1.4.1+), the AppImage delta threshold (v2.3.0), dependency versions, and video titles. zensical build --strict is clean and every internal link and anchor resolves. * Address review feedback - map-controls.md: the "Some tools call external services" warning claimed everything but Directions and Reverse Geocode ran locally, but Weather pulls overlay imagery from NASA and RainViewer. Named it in the warning. (Copilot and CodeRabbit both flagged this.) - features.md and index.md: replace "Print menu" with the Print Layout composer under Project, matching the actual UI. (Copilot) - adding-data.md and features.md: qualify the KML symbology claim. The in-house parser preserves embedded styling, but a file it cannot handle falls back to the DuckDB Spatial reader, which loads geometry without styling, as architecture.md:61 documents. (CodeRabbit) - architecture.md: correct the offline story for the CDN-loaded engines. The text claimed Pyodide and PGlite/PostGIS "are not offline-capable ... which the service worker does not cache", written in #280. PR #377 added the "geolibre-cdn-engines" CacheFirst rule (vite.config.ts) that caches exactly those jsDelivr URLs, so they do work offline after first online use. Also documents the Sedona and gdal3.js engines the rule covers, and notes the desktop build ships no service worker. * Address review feedback (round 2) - index.md: drop the duplicate "Print Layout composer" from the Map surface bullet; it is already covered under Media and capture. (Claude review) - architecture.md: the offline-setup paragraph listed four CDN engines but only explained how to remove the first-use network dependency for two. Adds GEOLIBRE_CEREUS_CDN=0 for the CereusDB wasm, and spells out that gdal3.js is the exception — it is never vendored, and GEOLIBRE_GDAL_CDN=0 opts out of the CDN rather than bundling it, leaving the Georeferencer's GeoTIFF/COG export unavailable. (CodeRabbit) * Address review feedback (round 3) - map-controls.md and features.md: the At Point / Pinned to Extent choice is not offered on every annotation. maplibre-annotations.ts:1107 gates the toggle on `type === "placed_image"`; everything else is point-anchored. Reworded both. (Claude review) - index.md and features.md: Cesium is a secondary-pane view, not "any map pane". MapGrid.tsx renders CesiumCanvas only inside SecondaryMapPane, and the primary map is always MapLibre. (CodeRabbit) - architecture.md: the sidecar section still listed Rasterio and SamGeo as future work, but raster.py, vector.py, and ml.py all ship. Describes what the sidecar does now and leaves Leafmap and GeoAI as the remaining work. (CodeRabbit) - architecture.md: "unless they are vendored" was too broad, since gdal3.js cannot be bundled. Names which engines can. (CodeRabbit) * Address review feedback (round 4) - projects.md and features.md: crash recovery is gated by `!isTauri() && !isEmbedded()` (useProjectHistory.ts:47), so the prompt is suppressed in embedded iframe sessions too, not just on desktop. Both notes now say so. (Claude review) - architecture.md: a self-hosted Pyodide mirror is not service-worker cached. The `geolibre-cdn-engines` rule matches only `cdn.jsdelivr.net` and the `geolibre-assets` rule requires same-origin `/assets/`, so a mirror on some other path falls back to ordinary HTTP caching. Split the paragraph so the two engines with no escape hatch (Pyodide, gdal3.js) are stated separately from the two that can be vendored. (CodeRabbit) * Address review feedback (round 5) - projects.md: the History dialog has no Discard action. ProjectHistoryDialog renders only `projectHistory.restore`; `projectHistory.discard` belongs to ProjectRecoveryDialog, the separate "Recover unsaved work?" prompt. Says instead that snapshots age out on their own. (Claude review) - map-controls.md: line width applies only to the stroked shapes. Arrow (its shaft goes through `lineFeature` -> `strokeProps`), Rectangle, Ellipse, and Freehand carry `stroke-width`; Text, Pin marker, Sticky note, and Placed image only carry a color. (Claude review) |
||
 Zahraandpre-commit-ci[bot]
|
c93327fc6f |
feat(add-data): geocode delimited-text addresses at import time (#1604)
* feat(add-data): geocode delimited-text addresses at import time Adds an "Addresses (geocode)" import mode to the Delimited Text Layer panel alongside the existing coordinate-column mode: pick one or more columns to concatenate into an address and each row is geocoded through the project's configured provider. Matched rows become points stamped with geocode_provider/geocode_status; unmatched rows are kept as null-geometry features (geocode_status: "unmatched") instead of being silently dropped, so they stay visible and fixable in the attribute table. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(add-data): address CSV-geocode review findings - Distinguish a batch cancelled before any row completed from a completed run with zero matches, so the thrown error doesn't misreport "No rows could be geocoded" for a cancellation. - Persist geocodeSkippedEmpty/geocodeCancelled on the layer alongside the existing geocodeMatched/geocodeUnmatched metadata, so a cancelled or partially-skipped run isn't silently indistinguishable from a complete one after the fact. - Give the address-columns empty-state hint its own i18n string instead of reusing the "Retrieve columns" button label as body copy. - Apply the unmatched-rows attribute filter after addAndClose selects the new layer, not before. * fix(add-data): fail fast on a missing API key, style geocoded points - Check geocoderNeedsApiKey before starting the batch, mirroring GeocodeDialog's guard, instead of paying a full paced loop of requests that are all doomed to fail with a keyed provider that has no key configured. - Pass the matched-feature collection as createBaseLayer's vector option so a geocoded address layer gets the as-added palette color and point sizing (#1519) like every other point-producing import path, instead of always rendering flat DEFAULT_LAYER_STYLE. An all-unmatched run (nothing to draw) keeps the flat defaults, mirroring the coordinate-mode attribute-table branch. * fix(add-data): preserve row order and stop stale address columns - Build the geocoded FeatureCollection in original CSV row order instead of grouping every matched point before every unmatched row, so the attribute table lines up with the source file. matchedFeatures now feeds the as-added style inference on its own, not the mixed collection. - Give the address-columns checklist its own field list derived only from the retrieved header, instead of reusing the coordinate-mode Select's field options, which always carries non-blank "longitude"/ "latitude" defaults and both hid the "retrieve columns first" empty state and could offer columns the file doesn't have. - Clear delimitedTextAddressColumns whenever the header resets (new file, delimiter change, mode switch) so a selection can't outlive the header it was picked against, and drop any stale selection that still slips through right before building geocode requests. * style: auto-format (ruff + oxfmt) [pre-commit.ci] * fix(add-data): clear the stale unmatched attribute filter setAttributeFilter was only ever set to "unmatched", never cleared, so a prior geocode run that left the global filter on "unmatched" stayed active and hid every row of a later, fully-matched layer. Always set it (to "unmatched" or "") after addAndClose selects the new layer. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Qiusheng Wu
|
c3510460a7 |
feat(planetary): USGS basemaps for nine more celestial bodies (#1196)
* feat(planetary): USGS basemaps for nine more celestial bodies Add Mercury, Venus, the four Galilean moons, Titan, Pluto, and Charon as selectable planetary basemaps. USGS Astrogeology only serves these as equirectangular WMS, so the tiles Worker now reprojects each tile to Web Mercator on the fly. Group the new bodies under one collapsible "Other celestial bodies" section, and make the on-map scale bar use the active body's radius instead of assuming Earth. * Address Claude review feedback - WMS path: skip negative-caching 5xx/429 upstream failures so a transient USGS outage/throttle isn't pinned as a blank tile, matching the OPM path. - Cap the /wms reprojection zoom (MAX_WMS_ZOOM) so a direct client can't run up Worker CPU + USGS load with unlimited high-z cache keys. - Add a [limits] cpu_ms block to wrangler.toml so the CPU-bound reprojection has a concrete budget and the code comment references a real setting. * Address CodeRabbit review feedback - Use Object.hasOwn() for both the OPM and WMS dataset lookups (ES2022, clearer than the hasOwnProperty.call form). * Address CodeRabbit review feedback - Replace the hand-rolled upng-js ambient declaration with the published @types/upng-js, switching to a namespace import; verified the reprojection still decodes/encodes correctly at runtime. * feat(planetary): use Venus MDIR colour in the planet switcher The Layers-panel celestial-body switcher now applies the Magellan C3-MDIR colour mosaic for Venus instead of the grayscale FMAP. * Address Claude review feedback - Lower MAX_WMS_ZOOM from 10 to 8 (one level above the datasets' native max) to shrink the high-zoom cache-key space a client could enumerate. - Log WMS reprojection failures (dataset/status/content-type) so a misconfigured dataset surfaces in Worker logs instead of failing silently as a blank tile. - Add tests asserting the picker groups: the nine non-Earth/Moon/Mars bodies land in the "other" section and it is sorted alphabetically by body name. - CollapsibleSection: pair aria-expanded with aria-controls via useId. * docs: document the expanded celestial-body basemaps Update the README, docs index, roadmap, and Adding Data guide to cover the new USGS Astrogeology basemaps (Mercury, Venus, the Galilean moons, Titan, Pluto, Charon) reprojected to Web Mercator, alongside the existing OpenPlanetaryMap Mars and Moon sets, the per-project ellipsoid, and the planet switcher. * Address Claude review feedback - Refresh the body-aware scale bar in applyMapPreferences, so changing the ellipsoid via Settings (without changing the basemap) updates the bar immediately instead of waiting for the next pan/zoom. * feat(planetary): auto-expand the Other bodies section for the active basemap The "Other celestial bodies" picker section stays collapsed by default but now opens automatically when the currently-active (or selected) basemap is one of those bodies, so the highlighted selection is never hidden. * Address Claude review feedback - Negative-cache WMS decode/warp/encode failures as a transparent tile (with a logged warning) instead of returning an uncached 502, so a misconfigured dataset degrades once rather than re-running the CPU-bound path forever. |
||
|
Qiusheng Wu
|
f8489a2b1a |
Add Google Photorealistic 3D Tiles support (#1046)
* Add Google Photorealistic 3D Tiles support
* Address Copilot, Claude, and CodeRabbit review feedback
- runtime-env.ts: correct the getGoogleMapsApiKey JSDoc. The bare GOOGLE_MAPS_API_KEY is not exposed via Vite envPrefix; it is reached via the vite.config build-time copy and window.__GEOLIBRE_RUNTIME_ENV__.
- vite.config.ts: loadEnv the app .env* files before the Google key shim so a key in .env.local also feeds VITE_GOOGLE_MAPS_API_KEY.
- 3d-tiles: register the Google submit interceptor on the panel with capture so it runs before the library form submit listener; covers both Load-button click and Enter submission.
- 3d-tiles: strip the masked X-GOOG-API-KEY header from the panel when the URL is switched away from a Google tileset, so the native submit path does not send or persist a bogus header.
- 3d-tiles: apply the altitude=0 default once with a dataset guard so a user deliberate -300 is not reset on every defaults pass.
- 3d-tiles: persist and apply the altitude offset for Google Photorealistic 3D Tiles by translating the deck.gl tile sublayer model matrices, including after-add input changes.
- 3d-tiles: warn when a submitted Google URL carries a key query parameter that is ignored in favor of the env or X-GOOG-API-KEY header.
- 3d-tiles: guard removeMapControl with try/catch; retry addMapControl on the next animation frame when it returns false; tear down the deck overlay when the last Google layer is removed; skip setProps when the Google layer signature is unchanged so unrelated layer mutations do not rebuild and re-fetch the tileset.
- 3d-tiles: render Google Photorealistic 3D Tiles through an interleaved MapboxOverlay so deck.gl shares the MapLibre WebGL context and does not appear as a separate canvas above the basemap.
- 3d-tiles: force mercator while Google tiles are present, restore the prior projection on removal, and persist the Google fly-to camera so GeoLibre view sync does not snap back to the default view.
- 3d-tiles: move the Google key reveal button outside the generated Request headers field so it does not crowd the following form controls.
* Fix CI build errors and address second review round
CI (clean build): the branch failed `tsc -b` with two errors a stale local
.tsbuildinfo had masked:
- Guard `getThreeDTilesUrlInput(panel)?.value` (string | undefined) before
passing to isGooglePhotorealisticTilesetUrl (moot after removing the
offending function below, but the direct guard is kept for the add path).
- Decouple RenderableDeckLayer from the abstract deck.gl `Layer` type so the
runtime Tile3DLayer subclass is not required to implement abstract members.
Review round 2:
- Altitude field no longer mutates every persisted Google layer live
(CodeRabbit Major): removed the altitude `input` listener and the
updateGooglePhotorealisticTilesAltitudeOffset helper; the add flow already
captures the altitude onto the new layer only. This also removes the
header-re-sanitize concern that helper raised.
- numberInputValue: treat an empty/whitespace field as the fallback instead of
resolving Number("") to 0.
- Render signature now includes header values (not just names) so a changed
custom header rebuilds the Tile3DLayer.
- Per-layer opacity slider no longer breaks mid-drag: the panel list only
rebuilds when the SET of Google layer ids changes, not on every store
mutation.
- Register the runtime-env-change listener from the Google overlay lifecycle so
a Google-only project (which never creates the native control) still picks up
an API-key change.
- Update googleTilesBoundMap on the async retry mount so a later ensure() does
not needlessly tear down and re-fetch the tileset.
- Latch the "gave up mounting" warning so it logs once, not on every render.
- Only capture "globe" as the pre-Google projection, never a forced/persisted
"mercator", so removing the last Google layer cannot leave the map stuck.
- Add a unit test for getGoogleMapsApiKey (VITE_/bare precedence, trimming).
- vite.config: resolve Vite's `--mode` from argv (fallback NODE_ENV) so
loadEnv reads mode-specific .env files.
|
||
 Qiusheng Wuandgiswqs
|
6c9f26dc1a |
docs: add User Guide and Tutorials (complete v1.0 documentation roadmap) (#194)
* docs: add User Guide and Tutorials, completing the v1.0 documentation roadmap Add a comprehensive feature-by-feature User Guide (interface, projects, adding data, layers, styling, attribute table, controls, processing, SQL workspace, data integrations, plugins, settings, embedding) and seven end-to-end Tutorials. Restructure the docs nav into User Guide, Tutorials, and Reference sections, link the new sections from the home page, and check off the final v1.0 roadmap item. Screenshots are hosted on Cloudflare R2 (data.geolibre.app/images) and referenced by URL, not committed to the repo. * docs: note the live demo is GitHub Pages-hosted, browser-only, and does not track users * docs: make sidebar section headers stand out (User Guide, Tutorials, Reference) * docs: align README and User Guide/Tutorials for consistency - Link the hosted User Guide and Tutorials from the README Documentation section - Standardize menu navigation to the arrow separator used by the rest of the docs - Fix SQL Workspace examples to use the dataset's real columns (NAME, POP_EST, GDP_MD_EST) and geometry column (geom), and bare-URL FROM, matching the in-app sample queries and the README * docs: refresh 'What GeoLibre does today' cards to current functionality Drop the removed sample-data plugin and surface the plugin marketplace; replace the stale 'roadmap toward DuckDB Spatial and a sidecar' processing card with the shipped Conversion menu and Whitebox toolbox; broaden the data-format and map-workspace cards to current sources, data-driven styling, sharing, and the on-map component tools. * chore: release v1.0.0 Bump all workspace packages, the Tauri desktop app (package.json, tauri.conf.json, Cargo.toml/Cargo.lock), and the Python sidecar to 1.0.0, and sync package-lock.json. Update the docs and README for the v1.0 release: mark v1.0 as current on the roadmap, retitle the README features section, rewrite the home page Project status for a stable v1.0, and refresh the v0.9.0 version labels in the architecture and project-format pages and the plugin manifest examples. * docs: add screenshots to text-only User Guide and tutorial pages Add dark-theme screenshots to Projects, Managing Layers, Styling Layers, Settings, Data Integrations, and Embedding & Sharing (every User Guide page now has at least one), plus the Add Vector panel in the Your First Map tutorial. Hosted on R2 and referenced by URL. * Address Claude and CodeRabbit review feedback - settings.md, projects.md, sharing-embedding.md: move the Share.GeoLibre API token to the Environment Variables section (it lives in the Environment tab, not Project Settings) and use the real field name; list the read-only Project file/format fields; fix "Restrict map bounds" and "Use Current View" labels - embedding.md, sharing-embedding.md: add iframe title and allow="fullscreen; geolocation"; escape & as & in iframe src attributes - embedding.md: note icon and icon-only as toolbar aliases - interface.md: separate the rotate (right-drag) and tilt (Ctrl/Cmd-drag) gestures - processing.md: note the menu order differs from the page grouping; note Union drops attributes; mark Vector to FlatGeobuf as Hilbert-sorted - cloud-native-data.md: clarify the Stream GeoParquet hint (spatial/Hilbert row-group sorting, not a bounding-box column) - plugins.md: manifests allow HTTPS or HTTP on loopback, not HTTPS-only - spatial-sql.md: note the sample columns are Natural Earth fields - extra.css: comment that the nav selectors target Material internals (9.x) * Address second round of Claude review feedback - spatial-sql.md: note ST_Area returns square degrees on EPSG:4326 data (ranking is valid; use ST_Area_Spheroid for km2) - spatial-sql.md: clarify the bare-URL FROM shorthand is a SQL Workspace convenience, not standard DuckDB CLI syntax * Address third round of Claude review feedback - spatial-sql.md: note the sample file's geometry column is `geom` and show DESCRIBE SELECT * FROM 'url' to discover columns in other datasets * Address fourth round of Claude review feedback - vector-analysis.md: mirror the Union "attributes are not preserved" caveat from processing.md (verified: both the Turf.js client and the GeoPandas sidecar drop attributes for Union) * Address Claude review feedback (round 6) - processing.md: note Intersection merges attributes from both layers and Difference keeps input attributes (verified in vector-tools.ts) - data-integrations.md: move GeoAgent out of "Time series and comparison" into its own "AI analysis" section - map-controls.md: correct the tilt gesture description (Ctrl/Cmd-drag tilts into a perspective view, not a 3D look-around mode) - adding-data.md: note that GeoParquet Layer opens the same Add Vector panel as Vector Layer * Address review feedback: soften the live-demo "secure" claim - embedding.md, index.md: replace the absolute "it is secure" assertion with concrete, verifiable statements (no analytics, no server account, client-side processing, no tracking), per maintainer decision and two bot reviews * Address Claude review feedback: sync Union note with the tutorial - processing.md: add "on either engine" to the Union attribute note so it matches the wording in vector-analysis.md --------- Co-authored-by: giswqs <giswqs@users.noreply.github.com> |