Garrick Aden-Buie
1acae30e83
re-run process workflow
vor 2 Jahren
Garrick Aden-Buie
162c446a75
update reports
vor 2 Jahren
Garrick Aden-Buie
4181058483
capture renv for each project
vor 2 Jahren
Garrick Aden-Buie
d22124428c
add final report doc
vor 2 Jahren
Garrick Aden-Buie
77e8a35f60
fix voters
vor 2 Jahren
Garrick Aden-Buie
ec1136e129
try to fix voters parquet file
vor 2 Jahren
Garrick Aden-Buie
0f340597d4
add snowflake queries
vor 2 Jahren
Garrick Aden-Buie
5e1312d55b
add delivery script
vor 2 Jahren
Garrick Aden-Buie
4fa4533c92
latest run
vor 2 Jahren
Garrick Aden-Buie
be5f430ffe
replace missing values with "" in receipts
vor 2 Jahren
Garrick Aden-Buie
df14b4203a
rebuild outputs
vor 2 Jahren
Garrick Aden-Buie
2ed69c1a11
pull latest reports
vor 2 Jahren
Garrick Aden-Buie
e26a8d48a1
fix candidate deduping by contest name
vor 2 Jahren
Garrick Aden-Buie
769eb5546d
move around unsued code
vor 2 Jahren
Garrick Aden-Buie
77cb0758bb
fill in missing candidate address with committee's address
vor 2 Jahren
Garrick Aden-Buie
051b60b922
fixups and full run
vor 2 Jahren
Garrick Aden-Buie
d94e89f37f
out voters, tweaks to receipts and committees
vor 2 Jahren
Garrick Aden-Buie
0a13f0ca90
out: candidate listing and officers
vor 2 Jahren
Garrick Aden-Buie
1775eb0526
clean up code
vor 2 Jahren
Garrick Aden-Buie
91ecf9eb62
normalized candidate listing (the hard way)
vor 2 Jahren
Garrick Aden-Buie
139d69eea7
repub donors
vor 2 Jahren
Garrick Aden-Buie
f269b4914e
snapshot
vor 2 Jahren
Garrick Aden-Buie
5655a2961a
out: addresses
vor 2 Jahren
Garrick Aden-Buie
8d9ac72fc5
work snapshot
vor 2 Jahren
Garrick Aden-Buie
e1dcb0667f
update reports
vor 2 Jahren
Garrick Aden-Buie
35f3a9a847
sort report list by report_id
vor 2 Jahren
Garrick Aden-Buie
5478feb4c0
easier method to load prepped data from parquet to duckdb tables
vor 2 Jahren
Garrick Aden-Buie
23690d6c76
progress before pause
vor 2 Jahren
Garrick Aden-Buie
1f8f26ce30
task: table of committees
vor 2 Jahren
Garrick Aden-Buie
7901dc4920
break process into two steps: prepare and process; pick final report
vor 2 Jahren
Garrick Aden-Buie
2ae2575005
fix: save result of table post-processing
vor 2 Jahren
Garrick Aden-Buie
422727b68d
need to know fixed sboe_id when writing out the parquet files
vor 2 Jahren
Garrick Aden-Buie
070e95a120
fix missing sboe_id values that are "No Id" in the database
By filling these in with `NOID-{report_id}`. This creates a unique sboe_id
for the committee for the report, otherwise all "No Id" committees would
end up being impossible to differentiate.
vor 2 Jahren
Garrick Aden-Buie
9a1c6642e3
add validation/exploration script
vor 2 Jahren
Garrick Aden-Buie
31ab569fd8
add `cf_db_create()`
vor 2 Jahren
Garrick Aden-Buie
87bca76b78
`cover` table should have distinct rows, fill in missing covers
vor 2 Jahren
Garrick Aden-Buie
9aa81ea812
faster data collection setup
vor 2 Jahren
Garrick Aden-Buie
120702d2ae
track collect/data-raw/report_list.csv
vor 2 Jahren
Garrick Aden-Buie
7a2cef064a
fix a bug in report processing
transposing the list would silently drop list elements
vor 2 Jahren
Garrick Aden-Buie
8be009d5cf
move reports into subdirs
vor 2 Jahren
Garrick Aden-Buie
fac27f746b
status and collection process update reports
vor 2 Jahren
Garrick Aden-Buie
f6e8c9deda
finish process
vor 2 Jahren
Garrick Aden-Buie
7b988094ca
read in report exports
vor 2 Jahren
Garrick Aden-Buie
8593c43fbe
process raw data in a new project
vor 2 Jahren
Garrick Aden-Buie
70369dd98b
move data collection into subfolder
vor 2 Jahren
Garrick Aden-Buie
c87a804c04
getting receipts and expenditures worked out
vor 2 Jahren
Garrick Aden-Buie
23d145fb5d
trying to parse the badly formatted csvs
vor 2 Jahren
Garrick Aden-Buie
df7d8d347b
prepping to read into parquet format
vor 2 Jahren
Garrick Aden-Buie
3f6def269c
reorganize targets an ensure up to date
vor 2 Jahren
Garrick Aden-Buie
3299e98daf
ignore data-raw folder
vor 2 Jahren