[ENG-8064] Add New Notifications Data Model (Refactor Notifications Phase 2)

[Johnetordoff]

Purpose

This system is designed to formalize and consolidate OSF Notifications under one system in order to better facilitate collaboration between Product, Engineers and QA and end persistent problems with notification email related tech debt. This project is the result of an extensive audit of all OSF emails and combined email digests and seeks to move the NotificationSubscription model out of it's transitional state having been migrated from a document based model, into it's final data model which is more regular for a relation our current relational database (django's postgress db).

In order to do this I have taken @mfraezz design documents and implemented them with minor changes. This means the responsibility for sending notifications in osf.io is shared by three new models, which will have the old data migrated into them via a migration script and management command. The models are:

• NotificationType

  • Similar to RegistrationSchemas or Waffle flags these are in db instrances which are poulated from static config documents in this case yaml.
  • Since these are synced on migration with notification.yaml a developer will be able to see at a glance if where a notification template is and what it's purpose is.

• NotificationSubscription

  • This model is somewhat analogous to the earlier EmailDigest model, this holds references to potentially many Notifcations models that can be compiled into a single digest this is sent at a periodic basis.

• Notification

  • Holds message information and context
  • Unlike earlier implementations this will record each Notification creation and sending for metrics purposes.

Changes

• Combines worker with beat in docker-compose
• creates notifications.yaml to contain all notification types info it is populated via postmigrate hook
• creates new Notification NotificationSubscription and NotificationType
• adds migrations to rename legacy tables and a managment command to populate the new ones.
• Deletes old send_mails method and replaces it with emit of NotificationType
• adds tests and updates old mocking
• updates views and permission classed
• A slight change to handle_boa_error to pass the already decanted user object.
• adds new data model for Notification, NotificationTypes and Subscriptions
• creates a notifications.yaml for data dependency notificationtypes
• add migrations
• updates notifications to use NotificationTypes
• updates Admin app to control email templates
• updates metrics reporter to count notifications sent etc.
• updates tests to all use capture_notifications mocking util
• Removes queued_mail
• Removes EmailDigest
• Removes detect_duplicates for duplicate subscriptions

QA Notes

Please make verification statements inspired by your code and what your code touches.

• Verify
• Verify

What are the areas of risk?

Any concerns/considerations/questions that development raised?

Documentation

Side Effects

Ticket

https://openscience.atlassian.net/browse/ENG-8064

[Johnetordoff]

@mfraezz I checked with another member of the cloud team, but just want to confirm with you this isn't still being used somewhere.

[Johnetordoff]

This gets added on demand now

[Johnetordoff]

@Johnetordoff Can I remove this?