Request For Comments (RFCs)
  • Request for comments (RFC)
  • RFC 001: Matcher architecture
  • RFC 002: Archival Storage Service
  • RFC 003: Asset Access
  • RFC 004: METS Adapter
  • RFC 005: Reporting Pipeline
  • RFC 006: Reindexer architecture
  • RFC 007: Goobi Upload
  • RFC 008: API Filtering
  • RFC 009: AWS account setup
  • RFC 010: Data model
  • RFC 011: Network Architecture
  • RFC 012: API Architecture
  • RFC 013: Release & Deployment tracking
    • Deployment example
    • Version 1
  • RFC 014: Born digital workflow
  • RFC 015: How we work
    • Code Reviews
    • Shared Libraries
  • RFC 016: Holdings service
  • URL Design
  • Pipeline Tracing
  • Platform Reliability
    • CI/CD
    • Observability
    • Reliability
  • RFC 020: Locations and requesting
  • RFC 021: Data science in the pipeline
  • RFC 022: Logging
    • Logging example
  • RFC 023: Images endpoint
  • RFC 024: Library management
  • RFC 025: Tagging our Terraform resources
  • RFC 026: Relevance reporting service
  • RFC 026: Relation Embedder
  • RFC 027: Pipeline Intermediate Storage
  • RFC 029: Work state modelling
  • Pipeline merging
  • RFC 031: Relation Batcher
  • RFC 032: Calm deletion watcher
  • RFC 033: Api internal model versioning
  • RFC 034: Modelling Locations in the Catalogue API
  • RFC 035: Modelling MARC 856 "web linking entry"
  • RFC 036: Modelling holdings records
  • API faceting principles & expectations
  • Matcher versioning
  • Requesting API design
  • TEI Adapter
  • Tracking changes to the Miro data
  • How do we tell users how to find stuff?
  • Removing deleted records from (re)indexes
  • RFC 044: Tracking Patron Deletions
  • Work relationships in Sierra, part 2
    • Work relationships in Sierra
  • Born Digital in IIIF
  • Transitive hierarchies in Sierra
  • RFC 047: Changing the structure of the Catalogue API index
  • RFC 048: Concepts work plan
  • RFC 049: Changing how aggregations are retrieved by the Catalogue API
  • RFC 050: Design considerations for the concepts API
  • RFC 051: Ingesting Library of Congress concepts
  • RFC: 052: The Concepts Pipeline - phase one
  • RFC 053: Logging in Lambdas
  • RFC 054: Authoritative ids with multiple Canonical ids.
  • RFC 055: Genres as Concepts
  • RFC 055: Content API
    • Content API: articles endpoint
    • Content API: Events endpoint
    • Content API: exhibitions endpoint
    • The future of this endpoint
  • RFC 056: Prismic to Elasticsearch ETL pipeline
  • RFC 57: Relevance testing
    • Examples of rank CLI usage
  • RFC 059: Splitting the catalogue pipeline Terraform
  • RFC 060: Service health-check principles
  • RFC 060: Offsite requesting
    • Sierra locations in the Catalogue API
  • Content-api: next steps
Powered by GitBook
On this page
  • Context
  • Proposal
  • Questions and potential issues

RFC 044: Tracking Patron Deletions

PreviousRemoving deleted records from (re)indexesNextWork relationships in Sierra, part 2

Last updated 10 months ago

Status: Draft

Last updated: 3/2/22

Context

We use Sierra as a source of truth for patron accounts: all account information is held in Sierra as a sole source of truth, and Sierra is used for all credential validation. For us, Auth0 effectively acts as the implementation for all of the OAuth2 roles, on top of Sierra.

While Sierra is the source of truth, Auth0 does store a minimal amount of patron data (email, barcode, patron number). We need to be able to get rid of the records in Auth0 when a record is removed from Sierra, for GDPR reasons as much as anything else.

Sierra does not provide a change feed or any other kind of notification for events, so we need a polling solution. We already do this for items and bibs , by polling for changes in a given time window. We do not need the complex architecture of the Sierra adapter for tracking patron deletions: we don't need any information from within the records, and we don't need to merge anything.

Proposal

  1. A CloudWatch event is run daily to trigger a Lambda

  2. The Lambda queries the Sierra API for patron records† that were deleted at some point on the previous day - so as to ensure the entire day is covered.

  3. The Lambda calls the to delete each of the records found in the query. If these are already deleted, or don't have any data in Auth0, this query 404s (which is fine).

† The Sierra API allows us to query by a deletedDate like yyyy-MM-dd, or an updatedDate like yyyy-MM-dd'T'HH:mm:ssZZ along with a deleted: true filter. While deletedDate has poor granularity, we should use it rather than updatedDate, as a deletion does not count as an update: the second option would not return records that were recently deleted but not otherwise updated.

Questions and potential issues

  • What happens if there are more deleted records than can be processed at once by the Lambda? Mitigation: increase Lambda timeout to maximum.

in the Sierra adapter
Auth0 Management API