Request For Comments (RFCs)
  • Request for comments (RFC)
  • RFC 001: Matcher architecture
  • RFC 002: Archival Storage Service
  • RFC 003: Asset Access
  • RFC 004: METS Adapter
  • RFC 005: Reporting Pipeline
  • RFC 006: Reindexer architecture
  • RFC 007: Goobi Upload
  • RFC 008: API Filtering
  • RFC 009: AWS account setup
  • RFC 010: Data model
  • RFC 011: Network Architecture
  • RFC 012: API Architecture
  • RFC 013: Release & Deployment tracking
    • Deployment example
    • Version 1
  • RFC 014: Born digital workflow
  • RFC 015: How we work
    • Code Reviews
    • Shared Libraries
  • RFC 016: Holdings service
  • RFC 017: URL Design
  • RFC 018: Pipeline Tracing
  • RFC 019: Platform Reliability
    • CI/CD
    • Observability
    • Reliability
  • RFC 020: Locations and requesting
  • RFC 021: Data science in the pipeline
  • RFC 022: Logging
    • Logging example
  • RFC 023: Images endpoint
  • RFC 024: Library management
  • RFC 025: Tagging our Terraform resources
  • RFC 026: Relevance reporting service
  • RFC 026: Relation Embedder
  • RFC 027: Pipeline Intermediate Storage
  • RFC 029: Work state modelling
  • RFC 030: Pipeline merging
  • RFC 031: Relation Batcher
  • RFC 032: Calm deletion watcher
  • RFC 033: Api internal model versioning
  • RFC 034: Modelling Locations in the Catalogue API
  • RFC 035: Modelling MARC 856 "web linking entry"
  • RFC 036: Modelling holdings records
  • RFC 037: API faceting principles & expectations
  • RFC 038: Matcher versioning
  • RFC 039: Requesting API design
  • RFC 040: TEI Adapter
  • RFC 041: Tracking changes to the Miro data
  • RFC 042: Requesting model
  • RFC 043: Removing deleted records from (re)indexes
  • RFC 044: Tracking Patron Deletions
  • RFC 045: Work relationships in Sierra, part 2
    • Work relationships in Sierra
  • RFC 046: Born Digital in IIIF
  • RFC 047: Changing the structure of the Catalogue API index
  • RFC 048: Concepts work plan
  • RFC 049: Changing how aggregations are retrieved by the Catalogue API
  • RFC 050: Design considerations for the concepts API
  • 051-concepts-adapters
  • RFC 052: The Concepts Pipeline - phase one
  • RFC 053: Logging in Lambdas
  • RFC 054: Authoritative ids with multiple Canonical ids.
  • RFC 055: Genres as Concepts
  • RFC 056: Prismic to Elasticsearch ETL pipeline
  • RFC 058: Relevance testing
    • Examples of rank CLI usage
  • RFC 059: Splitting the catalogue pipeline Terraform
  • RFC 060: Service health-check principles
  • RFC 061: Content API next steps
  • RFC 062: Content API: All search and indexing of addressable content types
  • RFC 062: Wellcome Collection Graph overview and next steps
  • RFC 063: Catalogue Pipeline services from ECS to Lambda
  • RFC 064: Graph data model
  • RFC 065: Library Data Link Explorer
  • RFC 066: Catalogue Graph pipeline
  • RFC 067: Prismic API ID casing
  • RFC 068: Exhibitions in Content API
  • RFC 069: Catalogue Graph Ingestor
  • RFC 070: Concepts API changes
  • RFC 071: Python Building and Deployment
    • The current state
  • RFC 072: Transitive Sierra hierarchies
  • RFC 073: Content API
    • Content API: articles endpoint
    • Content API: Events endpoint
    • Content API: exhibitions endpoint
    • The future of this endpoint
  • RFC 074: Offsite requesting
    • Sierra locations in the Catalogue API
  • RFC 075: Using Apache Iceberg tables in Catalogue Pipeline adapters
Powered by GitBook
On this page
  • Context
  • Proposal
  • Questions and potential issues

RFC 044: Tracking Patron Deletions

PreviousRFC 043: Removing deleted records from (re)indexesNextRFC 045: Work relationships in Sierra, part 2

Last updated 10 days ago

This RFC describes a proposal for tracking patron deletions in the Sierra API and removing corresponding records from Auth0.

Last modified: 2022-02-09T14:42:06+00:00

Context

We use Sierra as a source of truth for patron accounts: all account information is held in Sierra as a sole source of truth, and Sierra is used for all credential validation. For us, Auth0 effectively acts as the implementation for all of the OAuth2 roles, on top of Sierra.

While Sierra is the source of truth, Auth0 does store a minimal amount of patron data (email, barcode, patron number). We need to be able to get rid of the records in Auth0 when a record is removed from Sierra, for GDPR reasons as much as anything else.

Sierra does not provide a change feed or any other kind of notification for events, so we need a polling solution. We already do this for items and bibs , by polling for changes in a given time window. We do not need the complex architecture of the Sierra adapter for tracking patron deletions: we don't need any information from within the records, and we don't need to merge anything.

Proposal

  1. A CloudWatch event is run daily to trigger a Lambda

  2. The Lambda queries the Sierra API for patron records† that were deleted at some point on the previous day - so as to ensure the entire day is covered.

  3. The Lambda calls the to delete each of the records found in the query. If these are already deleted, or don't have any data in Auth0, this query 404s (which is fine).

† The Sierra API allows us to query by a deletedDate like yyyy-MM-dd, or an updatedDate like yyyy-MM-dd'T'HH:mm:ssZZ along with a deleted: true filter. While deletedDate has poor granularity, we should use it rather than updatedDate, as a deletion does not count as an update: the second option would not return records that were recently deleted but not otherwise updated.

Questions and potential issues

  • What happens if there are more deleted records than can be processed at once by the Lambda? Mitigation: increase Lambda timeout to maximum.

in the Sierra adapter
Auth0 Management API