lib.itmens/users/models.py
Henri Dickson 14b003a44a add all NeoDB features to NiceDB (#115)
* fix scraping failure with wepb image (merge upstream/fix-webp-scrape)

* add filetype to requirements

* add proxycrawl.com as fallback for douban scraper

* load 3p js/css from cdn

* add fix-cover task

* fix book/album cover tasks

* scrapestack

* bandcamp scrape and preview ;
manage.py scrape <url> ;
make ^C work when DEBUG

* use scrapestack when fix cover

* add user agent to improve compatibility

* search BandCamp for music albums

* add missing MovieGenre

* fix search 500 when song has no parent album

* adjust timeout

* individual scrapers

* fix tmdb parser

* export marks via rq; pref to send public toot; move import to data page

* fix spotify import

* fix edge cases

* export: fix dupe tags

* use rq to manage doufen import

* add django command to manage rq jobs

* fix export edge case

* tune rq admin

* fix detail page 502 step 1: async pull mastodon follow/block/mute list

* fix detail page 502 step 2: calculate relationship by local cached data

* manual sync mastodon follow info

* domain_blocks parsing fix

* marks by who i follows

* adjust label

* use username in urls

* add page to list a user\'s review

* review widget on user home page

* fix preview 500

* fix typo

* minor fix

* fix google books parsing

* allow mark/review visible to oneself

* fix auto sync masto for new user

* fix search 500

* add command to restart a sync task

* reset visibility

* delete user data

* fix tag search result pagination

* not upgrade to django 4 yet

* basic doc

* wip: collection

* wip

* wip

* collection use htmx

* show in-collection section for entities

* fix typo

* add su for easier debug

* fix some 500s

* fix login using alternative domain

* hide data from disabled user

* add item to list from detail page

* my tags

* collection: inline comment edit

* show number of ratings

* fix collection delete

* more detail in collection view

* use item template in search result

* fix 500

* write index to meilisearch

* fix search

* reindex in batch

* fix 500

* show search result from meilisearch

* more search commands

* index less fields

* index new items only

* search highlights

* fix 500

* auto set search category

* classic search if no meili server

* fix index stats error

* support typesense backend

* workaround typesense bug

* make external search async

* fix 500, typo

* fix cover scripts

* fix minor issue in douban parser

* supports m.douban.com and customized bandcamp domain

* move account

* reword with gender-friendly and instance-neutral language

* Friendica does not have vapid_key in api response

* enable anonymous search

* tweak book result template

* API v0

API v0

* fix meilisearch reindex

* fix search by url error

* login via twitter.com

* login via pixelfed

* minor fix

* no refresh on inactive users

* support refresh access token

* get rid of /users/number-id/

* refresh twitter handler automatically

* paste image when review

* support PixelFed (very long token)

* fix django-markdownx version

* ignore single quote for meilisearch for now

* update logo

* show book review/mark from same isbn

* show movie review/mark from same imdb

* fix login with older mastodon servers

* import Goodreads book list and profile

* add timestamp to Goodreads import

* support new google books api

* import goodreads list

* minor goodreads fix

* click corner action icon to add to wishlist

* clean up duplicated code

* fix anonymous search

* fix 500

* minor fix search 500

* show rating only if votes > 5

* Entity.refresh_rating()

* preference to append text when sharing; clean up duplicated code

* fix missing data for user tagged view

* fix page link for tag view

* fix 500 when language field longer than 10

* fix 500 when sharing mark for song

* fix error when reimport goodread profile

* fix minor typo

* fix a rare 500

* error log dump less

* fix tags in marks export

* fix missing param in pagination

* import douban review

* clarify text

* fix missing sheet in review import

* review: show in progress

* scrape douban: ignore unknown genre

* minor fix

* improve review import by guess entity urls

* clear guide text for review import

* improve review import form text

* workaround some 500

* fix mark import error

* fix img in review import

* load external results earlier

* ignore search server errors

* simplify user register flow to avoid inconsistent state

* Add a learn more link on login page

* Update login.html

* show mark created timestamp as mark time

* no 500 for api error

* redirect for expired tokens

* ensure preference object created.

* mark collections

* tag list

* fix tag display

* fix sorting etc

* fix 500

* fix potential export 500; save shared links

* fix share to twittwe

* fix review url

* fix 500

* fix 500

* add timeline, etc

* missing status change in timeline

* missing id in timeline

* timeline view by default

* workaround bug in markdownx...

* fix typo

* option to create new collection when add from detail page

* add missing announcement and tags in timeline home

* add missing announcement

* add missing announcement

* opensearch

* show fediverse shared link

* public review no longer requires login

* fix markdownx bug

* fix 500

* use cloudflare cdn

* validate jquery load and domain input

* fix 500

* tips for goodreads import

* collaborative collection

* show timeline and profile link on nav bar

* minor tweak

* share collection

* fix Goodreads search

* show wish mark in timeline

* resync failed urls with local proxy

* resync failed urls with local proxy: check proxy first

* scraper minor fix

* resync failed urls

* fix fields limit

* fix douban parsing error

* resync

* scraper minor fix

* scraper minor fix

* scraper minor fix

* local proxy

* local proxy

* sync default config from neodb

* configurable site name

* fix 500

* fix 500 for anonymous user

* add sentry

* add git version in log

* add git version in log

* no longer rely on cdnjs.cloudflare.com

* move jq/cash to _common_libs template partial

* fix rare js error

* fix 500

* avoid double submission error

* import tag in lower case

* catch some js network errors

* catch some js network errors

* support more goodread urls

* fix unaired tv in tmdb

* support more google book urls

* fix related series

* more goodreads urls

* robust googlebooks search

* robust search

* Update settings.py

* Update scraper.py

* Update requirements.txt

* make nicedb work

* doc update

* simplify permission check

* update doc

* update doc for bug report link

* skip spotify tracks

* fix 500

* improve search api

* blind fix import compatibility

* show years for movie in timeline

* show years for movie in timeline; thinner font

* export reviews

* revert user home to use jquery https://github.com/fabiospampinato/cash/issues/246

* IGDB

* use IGDB for Steam

* use TMDB for IMDb

* steam: igdb then fallback to steam

* keep change history

* keep change history: add django settings

* Steam: keep localized title/brief while merging IGDB

* basic Docker support

* rescrape

* Create codeql-analysis.yml

* Create SECURITY.md

* Create pysa.yml

Co-authored-by: doubaniux <goodsir@vivaldi.net>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Their Name <they@example.com>
Co-authored-by: Mt. Front <mfcndw@gmail.com>
2022-11-09 19:56:50 +01:00

183 lines
8 KiB
Python

import uuid
import django.contrib.postgres.fields as postgres
from django.db import models
from django.contrib.auth.models import AbstractUser
from django.utils import timezone
from django.core.serializers.json import DjangoJSONEncoder
from django.utils.translation import gettext_lazy as _
from common.utils import GenerateDateUUIDMediaFilePath
from django.conf import settings
from mastodon.api import *
def report_image_path(instance, filename):
return GenerateDateUUIDMediaFilePath(instance, filename, settings.REPORT_MEDIA_PATH_ROOT)
class User(AbstractUser):
if settings.MASTODON_ALLOW_ANY_SITE:
username = models.CharField(
_('username'),
max_length=150,
unique=False,
help_text=_('Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.'),
)
following = models.JSONField(default=list)
mastodon_id = models.CharField(max_length=100, blank=False)
# mastodon domain name, eg donotban.com
mastodon_site = models.CharField(max_length=100, blank=False)
mastodon_token = models.CharField(max_length=2048, default='')
mastodon_refresh_token = models.CharField(max_length=2048, default='')
mastodon_locked = models.BooleanField(default=False)
mastodon_followers = models.JSONField(default=list)
mastodon_following = models.JSONField(default=list)
mastodon_mutes = models.JSONField(default=list)
mastodon_blocks = models.JSONField(default=list)
mastodon_domain_blocks = models.JSONField(default=list)
mastodon_account = models.JSONField(default=dict)
mastodon_last_refresh = models.DateTimeField(default=timezone.now)
# store the latest read announcement id,
# every time user read the announcement update this field
read_announcement_index = models.PositiveIntegerField(default=0)
class Meta:
constraints = [
models.UniqueConstraint(
fields=['username', 'mastodon_site'], name="unique_user_identity")
]
# def save(self, *args, **kwargs):
# """ Automatically populate password field with settings.DEFAULT_PASSWORD before saving."""
# self.set_password(settings.DEFAULT_PASSWORD)
# return super().save(*args, **kwargs)
@property
def mastodon_username(self):
return self.username + '@' + self.mastodon_site
@property
def display_name(self):
return self.mastodon_account['display_name'] if self.mastodon_account and 'display_name' in self.mastodon_account and self.mastodon_account['display_name'] else self.mastodon_username
def __str__(self):
return self.mastodon_username
def get_preference(self):
pref = Preference.objects.filter(user=self).first() # self.preference
if not pref:
pref = Preference.objects.create(user=self)
return pref
def refresh_mastodon_data(self):
""" Try refresh account data from mastodon server, return true if refreshed successfully, note it will not save to db """
self.mastodon_last_refresh = timezone.now()
code, mastodon_account = verify_account(self.mastodon_site, self.mastodon_token)
if code == 401 and self.mastodon_refresh_token:
self.mastodon_token = refresh_access_token(self.mastodon_site, self.mastodon_refresh_token)
if self.mastodon_token:
code, mastodon_account = verify_account(self.mastodon_site, self.mastodon_token)
updated = False
if mastodon_account:
self.mastodon_account = mastodon_account
self.mastodon_locked = mastodon_account['locked']
if self.username != mastodon_account['username']:
print(f"username changed from {self} to {mastodon_account['username']}")
self.username = mastodon_account['username']
# self.mastodon_token = token
# user.mastodon_id = mastodon_account['id']
self.mastodon_followers = get_related_acct_list(self.mastodon_site, self.mastodon_token, f'/api/v1/accounts/{self.mastodon_id}/followers')
self.mastodon_following = get_related_acct_list(self.mastodon_site, self.mastodon_token, f'/api/v1/accounts/{self.mastodon_id}/following')
self.mastodon_mutes = get_related_acct_list(self.mastodon_site, self.mastodon_token, '/api/v1/mutes')
self.mastodon_blocks = get_related_acct_list(self.mastodon_site, self.mastodon_token, '/api/v1/blocks')
self.mastodon_domain_blocks = get_related_acct_list(self.mastodon_site, self.mastodon_token, '/api/v1/domain_blocks')
self.following = self.get_following_ids()
updated = True
elif code == 401:
print(f'401 {self}')
self.mastodon_token = ''
return updated
def get_following_ids(self):
fl = []
for m in self.mastodon_following:
target = User.get(m)
if target and ((not target.mastodon_locked) or self.mastodon_username in target.mastodon_followers):
fl.append(target.id)
return fl
def is_blocking(self, target):
return target.mastodon_username in self.mastodon_blocks or target.mastodon_site in self.mastodon_domain_blocks
def is_blocked_by(self, target):
return target.is_blocking(self)
def is_muting(self, target):
return target.mastodon_username in self.mastodon_mutes
def is_following(self, target):
return self.mastodon_username in target.mastodon_followers if target.mastodon_locked else self.mastodon_username in target.mastodon_followers or target.mastodon_username in self.mastodon_following
def is_followed_by(self, target):
return target.is_following(self)
def get_mark_for_item(self, item):
params = {item.__class__.__name__.lower() + '_id': item.id, 'owner': self}
mark = item.mark_class.objects.filter(**params).first()
return mark
def get_max_visibility(self, viewer):
if not viewer.is_authenticated:
return 0
elif viewer == self:
return 2
elif viewer.is_blocked_by(self):
return -1
elif viewer.is_following(self):
return 1
else:
return 0
@classmethod
def get(self, id):
if isinstance(id, str):
try:
username = id.split('@')[0]
site = id.split('@')[1]
except IndexError as e:
return None
query_kwargs = {'username': username, 'mastodon_site': site}
elif isinstance(id, int):
query_kwargs = {'pk': id}
else:
return None
return User.objects.filter(**query_kwargs).first()
class Preference(models.Model):
user = models.OneToOneField(User, models.CASCADE, primary_key=True)
home_layout = postgres.ArrayField(
postgres.HStoreField(),
blank=True,
default=list,
)
export_status = models.JSONField(blank=True, null=True, encoder=DjangoJSONEncoder, default=dict)
import_status = models.JSONField(blank=True, null=True, encoder=DjangoJSONEncoder, default=dict)
default_visibility = models.PositiveSmallIntegerField(default=0)
classic_homepage = models.BooleanField(null=False, default=False)
mastodon_publish_public = models.BooleanField(null=False, default=False)
mastodon_append_tag = models.CharField(max_length=2048, default='')
def get_serialized_home_layout(self):
return str(self.home_layout).replace("\'", "\"")
def __str__(self):
return str(self.user)
class Report(models.Model):
submit_user = models.ForeignKey(User, on_delete=models.SET_NULL, related_name='sumbitted_reports', null=True)
reported_user = models.ForeignKey(User, on_delete=models.SET_NULL, related_name='accused_reports', null=True)
image = models.ImageField(upload_to=report_image_path, height_field=None, width_field=None, max_length=None, blank=True, default='')
is_read = models.BooleanField(default=False)
submitted_time = models.DateTimeField(auto_now_add=True)
message = models.CharField(max_length=1000)