2023-11-27 20:04:15 +00:00
|
|
|
# WIP: descargador masivo de datos públicos
|
|
|
|
|
|
|
|
require [Node.js](https://nodejs.org) y [pnpm](https://pnpm.io/)
|
|
|
|
|
|
|
|
```
|
|
|
|
pnpm install
|
|
|
|
```
|
|
|
|
|
|
|
|
## correr
|
|
|
|
|
|
|
|
```
|
2023-11-28 02:19:48 +00:00
|
|
|
pnpm run run download_json.js https://datos.gob.ar/data.json
|
|
|
|
# guarda en ./datos.gob.ar
|
2023-11-27 20:04:15 +00:00
|
|
|
```
|
2023-11-28 03:44:47 +00:00
|
|
|
|
2023-11-28 21:38:40 +00:00
|
|
|
## contenedor
|
|
|
|
|
|
|
|
```
|
|
|
|
docker run --rm -it -e N_THREADS=128 -v ./data:/data gitea.nulo.in/nulo/transicion-desordenada-diablo/downloader
|
|
|
|
# descarga datos.gob.ar
|
|
|
|
```
|
|
|
|
|
2023-11-28 03:44:47 +00:00
|
|
|
## formato de repo guardado
|
|
|
|
|
|
|
|
- `{dominio de repo}`
|
|
|
|
- `data.json`
|
|
|
|
- `errors.jsonl`
|
|
|
|
- `{identifier de dataset}`
|
|
|
|
- `{identifier de distribution}`
|
|
|
|
- `{fileName (o, si no existe, identifier de distribution)}`
|