Преглед изворни кода

move data collection into subfolder

main
Garrick Aden-Buie пре 2 година
родитељ
комит
70369dd98b
No known key found for this signature in database
13 измењених фајлова са 13 додато и 10 уклоњено
  1. +3
    -2
      .gitignore
  2. +4
    -4
      collect/DESCRIPTION
  3. +0
    -0
      collect/R/get.R
  4. +6
    -4
      collect/R/read_report_file.R
  5. +0
    -0
      collect/R/urls.R
  6. +0
    -0
      collect/R/utils-delay.R
  7. +0
    -0
      collect/README.Rmd
  8. +0
    -0
      collect/_targets.R
  9. +0
    -0
      collect/_targets/.gitignore
  10. +0
    -0
      collect/_targets/meta/meta
  11. +0
    -0
      collect/nc-campaign-finance_collect.Rproj
  12. +0
    -0
      collect/run.R
  13. +0
    -0
      collect/scripts/2023-09-24_organize-data-raw-into-subfolders.R

+ 3
- 2
.gitignore Прегледај датотеку

.Rdata .Rdata
.history .history
0-time-log.csv 0-time-log.csv
data-raw/
data/
data-raw
collect/data-raw/
collect/data-old/

DESCRIPTION → collect/DESCRIPTION Прегледај датотеку

Type: Project Type: Project
Package: nc-campaign-finance
Title: What the Package Does (One Line, Title Case)
Package: nc-campaign-finance-collect
Title: Collect NC Campaign Finance Data
Version: 0.0.0.9000 Version: 0.0.0.9000
Authors@R: Authors@R:
person("Garrick", "Aden-Buie", , "garrick@adenbuie.com", role = c("aut", "cre"), person("Garrick", "Aden-Buie", , "garrick@adenbuie.com", role = c("aut", "cre"),
comment = c(ORCID = "0000-0002-7111-0077")) comment = c(ORCID = "0000-0002-7111-0077"))
Description: What the package does (one paragraph).
License: MIT + file LICENSE
Description: Collect campaign finance data from the NC State Board of Elections.
License: None
Encoding: UTF-8 Encoding: UTF-8
Roxygen: list(markdown = TRUE) Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3 RoxygenNote: 7.2.3

R/get.R → collect/R/get.R Прегледај датотеку


R/read_report_file.R → collect/R/read_report_file.R Прегледај датотеку



read_reports_by_sboe_id <- function(sboe_id) { read_reports_by_sboe_id <- function(sboe_id) {
raw_paths <- fs::dir_ls( raw_paths <- fs::dir_ls(
here::here("data-raw", "reports", sboe_id),
here::here("data-raw", "reports", sboe_id, "all"),
glob = "*.txt" glob = "*.txt"
) )
names(raw_paths) <- fs::path_rel(raw_paths, here::here("data-raw", "reports")) names(raw_paths) <- fs::path_rel(raw_paths, here::here("data-raw", "reports"))


data <-
raw_paths |>
raw_paths |>
map(read_report_file) |> map(read_report_file) |>
transpose() |> transpose() |>
map(list_rbind) |> map(list_rbind) |>
} }


report_path_info <- function(report_path) { report_path_info <- function(report_path) {
# data-raw/reports/{sboe_id}/all/{report_id}_{received_date}.txt
report_path <- fs::path_abs(report_path) report_path <- fs::path_abs(report_path)
x <- fs::path_rel(report_path, here::here("data-raw", "reports")) x <- fs::path_rel(report_path, here::here("data-raw", "reports"))
x <- fs::path_split(x)[[1]] x <- fs::path_split(x)[[1]]


id <- strsplit(x[2], "_")[[1]][1]
# x[2] is "all"

id <- strsplit(x[3], "_")[[1]][1]


list( list(
sboe_id = x[1], sboe_id = x[1],

R/urls.R → collect/R/urls.R Прегледај датотеку


R/utils-delay.R → collect/R/utils-delay.R Прегледај датотеку


README.Rmd → collect/README.Rmd Прегледај датотеку


_targets.R → collect/_targets.R Прегледај датотеку


_targets/.gitignore → collect/_targets/.gitignore Прегледај датотеку


_targets/meta/meta → collect/_targets/meta/meta Прегледај датотеку


nc-campaign-finance.Rproj → collect/nc-campaign-finance_collect.Rproj Прегледај датотеку


run.R → collect/run.R Прегледај датотеку


scripts/2023-09-24_organize-data-raw-into-subfolders.R → collect/scripts/2023-09-24_organize-data-raw-into-subfolders.R Прегледај датотеку


Loading…
Откажи
Сачувај