
* fix scraping failure with wepb image (merge upstream/fix-webp-scrape) * add filetype to requirements * add proxycrawl.com as fallback for douban scraper * load 3p js/css from cdn * add fix-cover task * fix book/album cover tasks * scrapestack * bandcamp scrape and preview ; manage.py scrape <url> ; make ^C work when DEBUG * use scrapestack when fix cover * add user agent to improve compatibility * search BandCamp for music albums * add missing MovieGenre * fix search 500 when song has no parent album * adjust timeout * individual scrapers * fix tmdb parser * export marks via rq; pref to send public toot; move import to data page * fix spotify import * fix edge cases * export: fix dupe tags * use rq to manage doufen import * add django command to manage rq jobs * fix export edge case * tune rq admin * fix detail page 502 step 1: async pull mastodon follow/block/mute list * fix detail page 502 step 2: calculate relationship by local cached data * manual sync mastodon follow info * domain_blocks parsing fix * marks by who i follows * adjust label * use username in urls * add page to list a user\'s review * review widget on user home page * fix preview 500 * fix typo * minor fix * fix google books parsing * allow mark/review visible to oneself * fix auto sync masto for new user * fix search 500 * add command to restart a sync task * reset visibility * delete user data * fix tag search result pagination * not upgrade to django 4 yet * basic doc * wip: collection * wip * wip * collection use htmx * show in-collection section for entities * fix typo * add su for easier debug * fix some 500s * fix login using alternative domain * hide data from disabled user * add item to list from detail page * my tags * collection: inline comment edit * show number of ratings * fix collection delete * more detail in collection view * use item template in search result * fix 500 * write index to meilisearch * fix search * reindex in batch * fix 500 * show search result from meilisearch * more search commands * index less fields * index new items only * search highlights * fix 500 * auto set search category * classic search if no meili server * fix index stats error * support typesense backend * workaround typesense bug * make external search async * fix 500, typo * fix cover scripts * fix minor issue in douban parser * supports m.douban.com and customized bandcamp domain * move account * reword with gender-friendly and instance-neutral language * Friendica does not have vapid_key in api response * enable anonymous search * tweak book result template * API v0 API v0 * fix meilisearch reindex * fix search by url error * login via twitter.com * login via pixelfed * minor fix * no refresh on inactive users * support refresh access token * get rid of /users/number-id/ * refresh twitter handler automatically * paste image when review * support PixelFed (very long token) * fix django-markdownx version * ignore single quote for meilisearch for now * update logo * show book review/mark from same isbn * show movie review/mark from same imdb * fix login with older mastodon servers * import Goodreads book list and profile * add timestamp to Goodreads import * support new google books api * import goodreads list * minor goodreads fix * click corner action icon to add to wishlist * clean up duplicated code * fix anonymous search * fix 500 * minor fix search 500 * show rating only if votes > 5 * Entity.refresh_rating() * preference to append text when sharing; clean up duplicated code * fix missing data for user tagged view * fix page link for tag view * fix 500 when language field longer than 10 * fix 500 when sharing mark for song * fix error when reimport goodread profile * fix minor typo * fix a rare 500 * error log dump less * fix tags in marks export * fix missing param in pagination * import douban review * clarify text * fix missing sheet in review import * review: show in progress * scrape douban: ignore unknown genre * minor fix * improve review import by guess entity urls * clear guide text for review import * improve review import form text * workaround some 500 * fix mark import error * fix img in review import * load external results earlier * ignore search server errors * simplify user register flow to avoid inconsistent state * Add a learn more link on login page * Update login.html * show mark created timestamp as mark time * no 500 for api error * redirect for expired tokens * ensure preference object created. * mark collections * tag list * fix tag display * fix sorting etc * fix 500 * fix potential export 500; save shared links * fix share to twittwe * fix review url * fix 500 * fix 500 * add timeline, etc * missing status change in timeline * missing id in timeline * timeline view by default * workaround bug in markdownx... * fix typo * option to create new collection when add from detail page * add missing announcement and tags in timeline home * add missing announcement * add missing announcement * opensearch * show fediverse shared link * public review no longer requires login * fix markdownx bug * fix 500 * use cloudflare cdn * validate jquery load and domain input * fix 500 * tips for goodreads import * collaborative collection * show timeline and profile link on nav bar * minor tweak * share collection * fix Goodreads search * show wish mark in timeline * resync failed urls with local proxy * resync failed urls with local proxy: check proxy first * scraper minor fix * resync failed urls * fix fields limit * fix douban parsing error * resync * scraper minor fix * scraper minor fix * scraper minor fix * local proxy * local proxy * sync default config from neodb * configurable site name * fix 500 * fix 500 for anonymous user * add sentry * add git version in log * add git version in log * no longer rely on cdnjs.cloudflare.com * move jq/cash to _common_libs template partial * fix rare js error * fix 500 * avoid double submission error * import tag in lower case * catch some js network errors * catch some js network errors * support more goodread urls * fix unaired tv in tmdb * support more google book urls * fix related series * more goodreads urls * robust googlebooks search * robust search * Update settings.py * Update scraper.py * Update requirements.txt * make nicedb work * doc update * simplify permission check * update doc * update doc for bug report link * skip spotify tracks * fix 500 * improve search api * blind fix import compatibility * show years for movie in timeline * show years for movie in timeline; thinner font * export reviews * revert user home to use jquery https://github.com/fabiospampinato/cash/issues/246 * IGDB * use IGDB for Steam * use TMDB for IMDb * steam: igdb then fallback to steam * keep change history * keep change history: add django settings * Steam: keep localized title/brief while merging IGDB * basic Docker support * rescrape * Create codeql-analysis.yml * Create SECURITY.md * Create pysa.yml Co-authored-by: doubaniux <goodsir@vivaldi.net> Co-authored-by: Your Name <you@example.com> Co-authored-by: Their Name <they@example.com> Co-authored-by: Mt. Front <mfcndw@gmail.com>
364 lines
10 KiB
Python
364 lines
10 KiB
Python
"""
|
|
Django settings for boofilsic project.
|
|
|
|
Generated by 'django-admin startproject' using Django 3.0.5.
|
|
|
|
For more information on this file, see
|
|
https://docs.djangoproject.com/en/3.0/topics/settings/
|
|
|
|
For the full list of settings and their values, see
|
|
https://docs.djangoproject.com/en/3.0/ref/settings/
|
|
"""
|
|
|
|
import os
|
|
import psycopg2.extensions
|
|
from git import Repo
|
|
|
|
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
|
|
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
# https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-auto-created-primary-keys
|
|
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
|
|
|
|
# Quick-start development settings - unsuitable for production
|
|
# See https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/
|
|
|
|
# SECURITY WARNING: keep the secret key used in production secret!
|
|
SECRET_KEY = 'nbv58c^&b8-095(^)&_BV98596v)&CX#^$&%*^V5'
|
|
|
|
# SECURITY WARNING: don't run with debug turned on in production!
|
|
DEBUG = True
|
|
|
|
ALLOWED_HOSTS = ['*']
|
|
|
|
# To allow debug in template context
|
|
# https://docs.djangoproject.com/en/3.1/ref/settings/#internal-ips
|
|
INTERNAL_IPS = [
|
|
"127.0.0.1"
|
|
]
|
|
|
|
# Application definition
|
|
|
|
INSTALLED_APPS = [
|
|
'django.contrib.admin',
|
|
'hijack',
|
|
'hijack.contrib.admin',
|
|
'django.contrib.auth',
|
|
'django.contrib.contenttypes',
|
|
'django.contrib.sessions',
|
|
'django.contrib.messages',
|
|
'django.contrib.staticfiles',
|
|
'django.contrib.humanize',
|
|
'django.contrib.postgres',
|
|
'django_sass',
|
|
'django_rq',
|
|
'simple_history',
|
|
'markdownx',
|
|
'management.apps.ManagementConfig',
|
|
'mastodon.apps.MastodonConfig',
|
|
'common.apps.CommonConfig',
|
|
'users.apps.UsersConfig',
|
|
'books.apps.BooksConfig',
|
|
'movies.apps.MoviesConfig',
|
|
'music.apps.MusicConfig',
|
|
'games.apps.GamesConfig',
|
|
'sync.apps.SyncConfig',
|
|
'collection.apps.CollectionConfig',
|
|
'timeline.apps.TimelineConfig',
|
|
'easy_thumbnails',
|
|
'user_messages',
|
|
'django_slack',
|
|
]
|
|
|
|
MIDDLEWARE = [
|
|
'django.middleware.security.SecurityMiddleware',
|
|
'django.contrib.sessions.middleware.SessionMiddleware',
|
|
'django.middleware.common.CommonMiddleware',
|
|
'django.middleware.csrf.CsrfViewMiddleware',
|
|
'django.contrib.auth.middleware.AuthenticationMiddleware',
|
|
'django.contrib.messages.middleware.MessageMiddleware',
|
|
'django.middleware.clickjacking.XFrameOptionsMiddleware',
|
|
'hijack.middleware.HijackUserMiddleware',
|
|
'simple_history.middleware.HistoryRequestMiddleware',
|
|
]
|
|
|
|
ROOT_URLCONF = 'boofilsic.urls'
|
|
|
|
TEMPLATES = [
|
|
{
|
|
'BACKEND': 'django.template.backends.django.DjangoTemplates',
|
|
'DIRS': [],
|
|
'APP_DIRS': True,
|
|
'OPTIONS': {
|
|
'context_processors': [
|
|
'django.template.context_processors.debug',
|
|
'django.template.context_processors.request',
|
|
'django.contrib.auth.context_processors.auth',
|
|
# 'django.contrib.messages.context_processors.messages',
|
|
"user_messages.context_processors.messages",
|
|
'boofilsic.context_processors.site_info',
|
|
],
|
|
},
|
|
},
|
|
]
|
|
|
|
WSGI_APPLICATION = 'boofilsic.wsgi.application'
|
|
|
|
|
|
# Database
|
|
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases
|
|
|
|
if DEBUG:
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.postgresql',
|
|
'NAME': os.environ.get('DB_NAME', 'test'),
|
|
'USER': os.environ.get('DB_USER', 'donotban'),
|
|
'PASSWORD': os.environ.get('DB_PASSWORD', 'donotbansilvousplait'),
|
|
'HOST': os.environ.get('DB_HOST', '172.18.116.29'),
|
|
'OPTIONS': {
|
|
'client_encoding': 'UTF8',
|
|
# 'isolation_level': psycopg2.extensions.ISOLATION_LEVEL_DEFAULT,
|
|
}
|
|
}
|
|
}
|
|
else:
|
|
DATABASES = {
|
|
'default': {
|
|
'ENGINE': 'django.db.backends.postgresql',
|
|
'NAME': 'boofilsic',
|
|
'USER': 'doubaniux',
|
|
'PASSWORD': 'password',
|
|
'HOST': 'localhost',
|
|
'OPTIONS': {
|
|
'client_encoding': 'UTF8',
|
|
# 'isolation_level': psycopg2.extensions.ISOLATION_LEVEL_DEFAULT,
|
|
}
|
|
}
|
|
}
|
|
|
|
# Customized auth backend, glue OAuth2 and Django User model together
|
|
# https://docs.djangoproject.com/en/3.0/topics/auth/customizing/#authentication-backends
|
|
|
|
AUTHENTICATION_BACKENDS = [
|
|
'mastodon.auth.OAuth2Backend',
|
|
]
|
|
|
|
|
|
# Internationalization
|
|
# https://docs.djangoproject.com/en/3.0/topics/i18n/
|
|
|
|
LANGUAGE_CODE = 'zh-hans'
|
|
|
|
TIME_ZONE = 'Asia/Shanghai'
|
|
|
|
USE_I18N = True
|
|
|
|
USE_L10N = True
|
|
|
|
USE_TZ = True
|
|
|
|
|
|
if not DEBUG:
|
|
SESSION_COOKIE_SECURE = True
|
|
CSRF_COOKIE_SECURE = True
|
|
SECURE_SSL_REDIRECT = True
|
|
SECURE_HSTS_PRELOAD = True
|
|
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
|
|
SECURE_HSTS_SECONDS = 31536000
|
|
|
|
LOGGING = {
|
|
'version': 1,
|
|
'disable_existing_loggers': False,
|
|
'formatters': {
|
|
'simple': {
|
|
'format': '{levelname} {asctime} {name}:{lineno} {message}',
|
|
'style': '{',
|
|
},
|
|
},
|
|
'handlers': {
|
|
'file': {
|
|
'level': 'INFO',
|
|
'class': 'logging.FileHandler',
|
|
'filename': os.path.join(BASE_DIR, 'log'),
|
|
'formatter': 'simple'
|
|
},
|
|
},
|
|
'root': {
|
|
'handlers': ['file'],
|
|
'level': 'INFO',
|
|
'propagate': True,
|
|
},
|
|
}
|
|
|
|
# Static files (CSS, JavaScript, Images)
|
|
# https://docs.djangoproject.com/en/3.0/howto/static-files/
|
|
|
|
STATIC_URL = '/static/'
|
|
STATIC_ROOT = os.path.join(BASE_DIR, 'static/')
|
|
|
|
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
|
|
|
|
AUTH_USER_MODEL = 'users.User'
|
|
|
|
SILENCED_SYSTEM_CHECKS = [
|
|
"auth.W004", # User.username is non-unique
|
|
"admin.E404" # Required by django-user-messages
|
|
]
|
|
|
|
MEDIA_URL = '/media/'
|
|
MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
|
|
|
|
PROJECT_ROOT = os.path.abspath(os.path.dirname(__name__))
|
|
SITE_INFO = {
|
|
'site_name': 'NiceDB',
|
|
'support_link': 'https://github.com/doubaniux/boofilsic/issues',
|
|
'version_hash': None,
|
|
'settings_module': os.getenv('DJANGO_SETTINGS_MODULE'),
|
|
'sentry_dsn': None,
|
|
}
|
|
|
|
# Mastodon configs
|
|
CLIENT_NAME = os.environ.get('APP_NAME', 'NiceDB')
|
|
SITE_INFO['site_name'] = os.environ.get('APP_NAME', 'NiceDB')
|
|
APP_WEBSITE = os.environ.get('APP_URL', 'https://nicedb.org')
|
|
REDIRECT_URIS = APP_WEBSITE + "/users/OAuth2_login/"
|
|
|
|
|
|
# Path to save report related images, ends with slash
|
|
REPORT_MEDIA_PATH_ROOT = 'report/'
|
|
MARKDOWNX_MEDIA_PATH = 'review/'
|
|
BOOK_MEDIA_PATH_ROOT = 'book/'
|
|
DEFAULT_BOOK_IMAGE = os.path.join(BOOK_MEDIA_PATH_ROOT, 'default.svg')
|
|
MOVIE_MEDIA_PATH_ROOT = 'movie/'
|
|
DEFAULT_MOVIE_IMAGE = os.path.join(MOVIE_MEDIA_PATH_ROOT, 'default.svg')
|
|
SONG_MEDIA_PATH_ROOT = 'song/'
|
|
DEFAULT_SONG_IMAGE = os.path.join(SONG_MEDIA_PATH_ROOT, 'default.svg')
|
|
ALBUM_MEDIA_PATH_ROOT = 'album/'
|
|
DEFAULT_ALBUM_IMAGE = os.path.join(ALBUM_MEDIA_PATH_ROOT, 'default.svg')
|
|
GAME_MEDIA_PATH_ROOT = 'game/'
|
|
DEFAULT_GAME_IMAGE = os.path.join(GAME_MEDIA_PATH_ROOT, 'default.svg')
|
|
COLLECTION_MEDIA_PATH_ROOT = 'collection/'
|
|
DEFAULT_COLLECTION_IMAGE = os.path.join(COLLECTION_MEDIA_PATH_ROOT, 'default.svg')
|
|
SYNC_FILE_PATH_ROOT = 'sync/'
|
|
EXPORT_FILE_PATH_ROOT = 'export/'
|
|
|
|
# Allow user to login via any Mastodon/Pleroma sites
|
|
MASTODON_ALLOW_ANY_SITE = False
|
|
|
|
# Timeout of requests to Mastodon, in seconds
|
|
MASTODON_TIMEOUT = 30
|
|
|
|
MASTODON_CLIENT_SCOPE = 'read write follow'
|
|
#use the following if it's a new site
|
|
#MASTODON_CLIENT_SCOPE = 'read:accounts read:follows read:search read:blocks read:mutes write:statuses write:media'
|
|
|
|
MASTODON_LEGACY_CLIENT_SCOPE = 'read write follow'
|
|
|
|
# Tags for toots posted from this site
|
|
MASTODON_TAGS = '#NiceDB #NiceDB%(category)s #NiceDB%(category)s%(type)s'
|
|
|
|
# Emoji code in mastodon
|
|
STAR_SOLID = ':star_solid:'
|
|
STAR_HALF = ':star_half:'
|
|
STAR_EMPTY = ':star_empty:'
|
|
|
|
# Default password for each user. since password is not used any way,
|
|
# any string that is not empty is ok
|
|
DEFAULT_PASSWORD = 'ab7nsm8didusbaqPgq'
|
|
|
|
# Default redirect loaction when access login required view
|
|
LOGIN_URL = '/users/login/'
|
|
|
|
# Admin site root url
|
|
ADMIN_URL = 'tertqX7256n7ej8nbv5cwvsegdse6w7ne5rHd'
|
|
|
|
# Luminati proxy settings
|
|
LUMINATI_USERNAME = 'lum-customer-hl_nw4tbv78-zone-static'
|
|
LUMINATI_PASSWORD = 'nsb7te9bw0ney'
|
|
|
|
SCRAPING_TIMEOUT = 90
|
|
|
|
# ScraperAPI api key
|
|
SCRAPERAPI_KEY = 'wnb3794v675b8w475h0e8hr7tyge'
|
|
PROXYCRAWL_KEY = None
|
|
SCRAPESTACK_KEY = None
|
|
|
|
# Spotify credentials
|
|
SPOTIFY_CREDENTIAL = "NzYzNkYTE6MGQ0ODY0NTY2Y2b3n645sdfgAyY2I1ljYjg3Nzc0MjIwODQ0ZWE="
|
|
|
|
# IMDb API service https://imdb-api.com/
|
|
IMDB_API_KEY = "k23fwewff23"
|
|
|
|
# The Movie Database (TMDB) API Keys
|
|
TMDB_API3_KEY = "deadbeef"
|
|
TMDB_API4_KEY = "deadbeef.deadbeef.deadbeef"
|
|
|
|
# Google Books API Key
|
|
GOOGLE_API_KEY = 'deadbeef-deadbeef-deadbeef'
|
|
|
|
# IGDB
|
|
IGDB_CLIENT_ID = 'deadbeef'
|
|
IGDB_ACCESS_TOKEN = 'deadbeef'
|
|
|
|
# Thumbnail setting
|
|
# It is possible to optimize the image size even more: https://easy-thumbnails.readthedocs.io/en/latest/ref/optimize/
|
|
THUMBNAIL_ALIASES = {
|
|
'': {
|
|
'normal': {
|
|
'size': (200, 200),
|
|
'crop': 'scale',
|
|
'autocrop': True,
|
|
},
|
|
},
|
|
}
|
|
# THUMBNAIL_PRESERVE_EXTENSIONS = ('svg',)
|
|
if DEBUG:
|
|
THUMBNAIL_DEBUG = True
|
|
|
|
# https://django-debug-toolbar.readthedocs.io/en/latest/
|
|
# maybe benchmarking before deployment
|
|
|
|
REDIS_HOST = os.environ.get('REDIS_HOST', '127.0.0.1')
|
|
|
|
RQ_QUEUES = {
|
|
'mastodon': {
|
|
'HOST': REDIS_HOST,
|
|
'PORT': 6379,
|
|
'DB': 0,
|
|
'DEFAULT_TIMEOUT': -1,
|
|
},
|
|
'export': {
|
|
'HOST': REDIS_HOST,
|
|
'PORT': 6379,
|
|
'DB': 0,
|
|
'DEFAULT_TIMEOUT': -1,
|
|
},
|
|
'doufen': {
|
|
'HOST': REDIS_HOST,
|
|
'PORT': 6379,
|
|
'DB': 0,
|
|
'DEFAULT_TIMEOUT': -1,
|
|
}
|
|
}
|
|
|
|
RQ_SHOW_ADMIN_LINK = True
|
|
|
|
SEARCH_INDEX_NEW_ONLY = False
|
|
|
|
SEARCH_BACKEND = None
|
|
|
|
# SEARCH_BACKEND = 'MEILISEARCH'
|
|
# MEILISEARCH_SERVER = 'http://127.0.0.1:7700'
|
|
# MEILISEARCH_KEY = 'deadbeef'
|
|
|
|
# SEARCH_BACKEND = 'TYPESENSE'
|
|
# TYPESENSE_CONNECTION = {
|
|
# 'api_key': 'deadbeef',
|
|
# 'nodes': [{
|
|
# 'host': 'localhost',
|
|
# 'port': '8108',
|
|
# 'protocol': 'http'
|
|
# }],
|
|
# 'connection_timeout_seconds': 2
|
|
# }
|