nc-campaign-finance

Garrick Aden-Buie 4181058483 capture renv for each project		2年前
..
R	process raw data in a new project	2年前
_targets	pull latest reports	2年前
data-raw	pull latest reports	2年前
renv	capture renv for each project	2年前
scripts	move data collection into subfolder	2年前
.Rprofile	capture renv for each project	2年前
DESCRIPTION	move data collection into subfolder	2年前
README.Rmd	move data collection into subfolder	2年前
_targets.R	sort report list by report_id	2年前
nc-campaign-finance_collect.Rproj	move data collection into subfolder	2年前
renv.lock	capture renv for each project	2年前
run.R	move data collection into subfolder	2年前

README.Rmd



https://cf.ncsbe.gov/CFDocLkup/DocumentResult/?year=2022&reports=%27RPQTR4%27

On election years: 2008, 2012, 2016, 2020
`https://cf.ncsbe.gov/CFDocLkup/DocumentResult/?year={year}&reports='RPQTR{quarter}'`

On non-election years:
```
https://cf.ncsbe.gov/CFDocLkup/DocumentResult/?year=2023&reports=%27RPMYSA%27%2C%20%27RPYESA%27
https://cf.ncsbe.gov/CFDocLkup/DocumentResult/?year=2023&reports=%27RPQTR1%27%2C%20%27RPQTR4%27%2C%20%27RPQTR2%27%2C%20%27RPQTR3%27
```


- `RPMYSA` - Mid-Year Semi-Annual Report
- `RPYESA` - Year-End Semi-Annual Report
- `RPQTR1` - 1st Quarter Report
- `RPQTR2` - 2nd Quarter Report
- `RPQTR3` - 3rd Quarter Report
- `RPQTR4` - 4th Quarter Report

```{r}
year <- 2022
reports <- "'RPQTR1', 'RPQTR4', 'RPQTR2', 'RPQTR3'"
url <- glue::glue("https://cf.ncsbe.gov/CFDocLkup/DocumentResult/?year={year}&reports={reports}")
```


## All documents by SBOE ID

Having all of the documents, we need the report ID for the reports listed above.

```
                                              v--------v
https://cf.ncsbe.gov/CFOrgLkup/ReportSection/?RID=197247&SID=STA-8S285O-C-001
```

That links to the sections of the report, where you can click the "HTML View" of "All report sections":

```
https://cf.ncsbe.gov/CFOrgLkup/ReportDetail/?RID=197247&TP=ALL
```

At the top of that page is a link to the reports csv:

```
# all reports
https://cf.ncsbe.gov/CFOrgLkup/ExportDetailResults/?ReportID=197247&Type=ALL
# report cover
https://cf.ncsbe.gov/CFOrgLkup/ExportDetailResults/?ReportID=197247&Type=CVR
# detailed receipts
https://cf.ncsbe.gov/CFOrgLkup/ExportDetailResults/?ReportID=197247&Type=REC
# detailed expenditures
https://cf.ncsbe.gov/CFOrgLkup/ReportDetail/?RID=197247&TP=EXP
```

## Trying yet again

It turns out that using the "Export data to CSV" link from a
[page like this one](https://cf.ncsbe.gov/CFOrgLkup/ReportDetail/?RID=205761&TP=ALL)
will return a very badly formatted CSV file, potentially full of errors.

On the other hand, we can the pieces via:

```
# Reports
# (paged, unclear page size)
https://cf.ncsbe.gov/CFOrgLkup/GetReceipts?ReportID={report_id}&page=0&pageSize=100

# Expenses
# (paged, unclear page size)
https://cf.ncsbe.gov/CFOrgLkup/GetExpenditures?ReportID={report_id}&page=0&pageSize=100&ShowIEColumns=true


```