- Use transformers to trim incoming strings at import that are too long for DB columns:
- Bill: document note, version note
- Event: media note
- This release was yanked. It involved a database migration that operationally failed, and was reverted.
- Add additional log info re: archiving scrape files to cloud storage
- Fix bug with import caused by newish "extras" field on scraper
- Improve resolve bill to capture at least the most recent session
- Fix matching committee organizations when chamber is specified for an organizational bill sponsor
- Add steps to publish lightweight openstates-metadata
- Improve Event to Person matching
- Add Chamber on Bill Sponsorship matches for people
- Improve Event to Bill matching for Events on last day of session
- Improve query for searching committee other_names
- Prevent failure in Google Cloud Storage archiving from failing a scrape/update operation
- Fix permissions issue caused by slightly wrong usage of GCP storage client code
- Adds support for --archive flag on os-update to archive a full scrape to google cloud storage bucket
- Add instructions on DB migrations to docs
- Change schema to fix two DB issues:
- Event Document
notechanged to Text type to avoid character limit errors - Vote Event
dedupe_keyindex added to improve performance on Vote Event lookups during import
- Event Document
- Turn off archiving processed realtime bills by default.
- Adds a new CLI tool that can be called to resolve unresolved bill-to-bill relationships in the openstates DB
- other_name in committee matching
- swap getargspec for getfullargspec on Update
- Rollback on committee matching
- Add other_names to committee matching for Event Imports
- Maine (ME): add a tribal representative district to the jurisdiction metadata
- Stop validating the repeated use of the same phone number in People offices data
- People: add distinct mailing classification options (district-mail, capitol-mail) and home classification to offices
- Docs: Add debug instructions for the update command by @jessemortenson in #119
- allow postimport hook to be skipped in os-update command
- set application_name for better performance monitoring
- Add new bill statuses to validation
- Small territory fixes
- update pyyaml dependency
- Support additional territories in db init
- Add additional US territory support#113
- update logs on import
- check abbr on division creation
- change name lookup
- Correct initdb for ZA
- Init metadata for ZA
- Add metadata for ZA
- Update bill action classification options
- Update MA metadata
- Change to force deploy
- Bump version correctly
- Fix CI bug
- Update legacy district ids & seats
- Add database migrations & metadata updates to release steps
- Fix boto3 library usage for realtime consumption
- Add CLI tool for manual JSON schema validation
- Updated Districts for NH
- Updated Districts for PR, MD, MA, NH, ID, ND, VT
- Realtime Bills Metrics Update
- Updated Districts for MA and VT
- skip bad tags
- revert nomination event type
- add nomination as valid event type
- update to metadata for MD
- update to metadata for MA, MD, ND, NH, US, VT, WV, WY
- add SQS for batching
- update us legacy districts
- update us district numbers
- check for congress bio info existing before assigning values
- properly handle bad UUIDs in event objects
- session key bug fix
- Actually set timeout on JWT tokens
- clean up instrumentation endpoint
- fix import bug
- emit stat data as scrapers run
- adds realtime bill yield
- update dependency
- decrease size of dockerfile
- update dependency
- add additional parties for NY senate
- allow returning None objects in scrapes
- skip objects with unresolved ID error
- update django dependency
- FL: Fix text extraction
- move scrapelib dependency to Civic Eagle forked repo
- add wikidata to people ids
- test for rylie
- loosen version requirements for spatula
- fix openstates/issues#598, committees with duplicate names
- update click dependency
- US text extraction enabled
- fix for openstates/issues#608
- fix typo in 6.10.3 release
- fix for subcommittee memberships not being ignored on person import
- special case fix for extracting bill text from California (openstates/issues#147)
- fixes for people merging
- handle null committees in committee merge/scrape
- allow executives to have inactive roles too for linting purposes
- add committees to os-scrape command
- improve committee merging
- add prototype os-scrape command to help with people updates
- people model fix for empty URL validation
- people model fix for new suffixes
- add --reset-offices flag for people merging when office data has changed significantly
- lots of improvements to os-people merge's automation
- improve committee membership validation
- add migration to os-initdb
- fix for bad phone numbers in update-us
- OSEP #2: add citation support to Bill
- several test fixes for OS committee changes
- fix for update-us
- os-committee linting improvements
- os-committee merge avoids duplicates
- fix for circular import in openstates.metadata
- OSEP #8: add LegislativeSession.active flags
- running a scraper without a session will now run active sessions instead of "latest"
- removal of BillScraper.latest_session in favor of new session injection
- replace scrape.Jurisdiction with scrape.State, which was formerly in scrapers.utils
- remove PersonContactDetail entirely, completing OSEP #6
- bugfix for openstates.data django app with new PersonOffice model
- bugfix for us-to-yaml addresses missing names
- implement core parts of OSEP #6: switch to Person.offices from Person.contact_details
- bugfixes for committee merge with joint committees & case-insensitive matching
- bugfix for EventImporter.postimport being called after bill-only scrapes
- bugfix for duplicates on Event importer
- fix for bill matching on Event importer
- speed up imports when there are no bills in import
- import fix for committee_id on scraped events
- enable import of events by default in os-update
- allow raising
EmptyScrapeto end a scrape without 'no objects returned' error - added
upstream_idto Event model - add classification parameter to
Event.add_document - fix up
Event.add_media_linkparameters to be uniform (including classification) - added
Event.add_billhelper method os-people-to-yamlis now part ofos-people merge- Events imports now include soft-deletion of events that are not included in latest scrape
- committee merge bugfix for parent committees
- normalize whitespace in committee names/members
- add --fix flag to committee lint
- fix os-us-to-yaml phone numbers
- added latest_bill_update and latest_people_update to Jurisdiction table (these fields are now automatically updated when os-update or os-people update the relevant tables)
- moved openstates.reports into openstates.data
- fix os-us-to-yaml with no office details
- fix for OS_PEOPLE_DIRECTORY setting detection
- fix committee import to include extras
- stop auto-purging committees
- add linting check for committee homepages
- committees will always have classification in file
- fix committee linting/import/etc. to set parent key to an ocd-organization ID
- fix for committee parent/subcommittee in ScrapeCommittee
- committees in os-us-to-yaml
- committees linting includes warnings now
- refactor openstates.people into openstates.models and openstates.utils
- refactor openstates.people into openstates.models and openstates.utils
- committee database imports
- add support for scraping subcommittees
- set charset on CSV uploads
- fix for ScrapeCommittee add_link/add_source
- improved use of typing, which will now be enforced going forward
- added --save-all flag to os-people lint
- us legislators script fixes
- fix boto being initialized if not used
- bugfix for os-people's get_all_abbreviations()
- bump to Python 3.9
- move openstates/text-extraction repository into openstates.fulltext
- move openstates/people tools into openstates.cli and openstates.people
- add obsolete US districts to metadata
- add US territories to metadata
- backwards incompatible migration needed: Organization fields JSON-ized (part of OSEP #4)
- updates to work with Django 3.2
- warnings on duplicate sponsors
- case-insensitive person matching
- improvement for person matching to restrict to legislative roles
- fix for organization sponsors on import
- fix for get_seen_sessions error introduced in 5.9.0
- improved Person resolution in importers
- added typing to openstates.metadata
- added typing to openstates.importers
- fix for microsecond replacement on date
- fix_bill_id fixed for US bills
- strip microseconds from datetimes in JSON output
- remaining implementation of OSEP #5
- remove pupa_id backwards compatibility
- migration: drop reports.Identifier
- Add veto and veto-override vote classifications
- bugfix for typo in scrape schema in 5.7.0
- Implementation of OSEP #5
- migration: added dedupe_key to replace pupa_id
- added dedupe_key to replace pupa_id in scrape as well
- alter import logic to stop using reports.Identifier
- add support for US jurisdiction
- add 'related' as additional bill relation type (openstates/issues#181)
- subjects are now sorted automatically in output JSON
- classifications are now sorted automatically in output JSON
- EventAgendaItem.order converted to be an integer (migration required)
- bugfix for openstates/issues#171 where updated_at is churning when there are two identical actions
- update PR senate at-large, 11 senators again
- update MA district metadata, openstates-core#18
- AZ & NM timezone updates from upstream python-us
- set Django time zone to UTC by default
- remove unused OrderVoteEvent class
- add top-level email field to Person
- add top-level email field to Person
- disallow bill duplicates that are in opposite chambers
- correct issues with jurisdiction and organization enums
- schema change: add BillDocument and BillVersion classification options
- fix Django 3.1 warnings
- fix Jurisdiction creation classification
- fix typo in admin.py for bills
- removed some bad field references
- merge openstates_metadata into openstates.metadata
- large database cleanup:
- remove unused fields:
- BillAbstract.date
- BillAction.extras
- BillIdentifier.scheme & note
- Jurisdiction.feature_flags
- Membership.label
- Organization.image, founding_date, dissolution_date
- Person.summary
- PersonContactDetail.label
- Post.start_date & end_date
- VoteEvent.end_date
- BillDocumentLink & BillVersionLink.text
- *.locked_fields
- add VoteEvent.order
- remove unused fields:
- remove GIS dependency, alter (unused) EventLocation model accordingly
- remove person_role_division_id
- fix Person.objects.active()
- make os-initdb idempotent (and test it!)
- schema change: add Person.current_jurisdiction_id & Person.current_role
- schema change: allow Jurisdictions to exist without Divisions
- fix clean_whitespace type issues
- add 'warn' behavior, now default, to add_version_link/add_document_link
- strip whitespace from all data when being saved
- re-enable VoteEvent classification validation
- start & end date required on legislative_sessions
- bugfix for member_of with post
- bugfix for os-update importer crashing
- bugfix for Person.member_of returning duplicates
- simplify org and person import code as part of refactor
- add Person.current_state
- alter Bill computed fields to be null if unset
- update Person queries to make use of new computed fields
- bugfix for os-update-computed
- bugfix for LegislativeSession in admin
- improve update computed command
- add BaseImporter.update_computed_fields
- fix return code
- make new fields temporarily nullable
- fix updated-computed init_django bug
- add computed fields on Person model
- add computed fields on Bill model
- Person.current_role no longer requires database lookups
- add os-updated-computed command that updates computed fields
- remove some completely unused fields & models
- add Person current_role, active, current_legislators_with_roles, and search helpers
- remove some unused importer code for Memberships and Posts
- create executive org on initdb
- allow for dynamic module to handle scrape output instead of JSON dump
- add os-initdb to get a database with core models in place
- remove old loaddivisions management command
- bugfixes for admin view URLs
- fix reports tablenames to match old pupa tablenames for now to keep backwards compatibility
-
first release of combined project - essentially without changes
Prior to 4.0, was two projects:
- https://github.com/opencivicdata/python-opencivicdata/blob/master/CHANGELOG.md
- https://github.com/opencivicdata/pupa/blob/master/CHANGELOG.md
Code in those projects is subject to the license files in the root of the repository.