Merge branch 'main' of https://github.com/neodb-social/neodb
This commit is contained in:
commit
3d11d03a01
GPG key ID:
6CE35141D31CEAFB
51 changed files with 1960 additions and 4080 deletions
16
.github/workflows/check.yml
vendored
16
.github/workflows/check.yml
vendored
|
|
@ -3,22 +3,6 @@ name: code check
|
||||||
on: [push, pull_request]
|
on: [push, pull_request]
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
doc:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
matrix:
|
|
||||||
python-version: ['3.12']
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
- name: Set up Python ${{ matrix.python-version }}
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ matrix.python-version }}
|
|
||||||
cache: pip
|
|
||||||
- name: Run pre-commit
|
|
||||||
run: |
|
|
||||||
python -m pip install mkdocs-material
|
|
||||||
mkdocs build -s -d /tmp/neodb-docs
|
|
||||||
lint:
|
lint:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
strategy:
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,3 @@
|
||||||
# Federation
|
# Federation
|
||||||
|
|
||||||
see [doc](docs/internals/federation.md) for FEP-67ff related information.
|
see [doc](https://neodb.net/internals/federation.md) for FEP-67ff related information.
|
||||||
|
|
|
||||||
14
README.md
14
README.md
|
|
@ -4,19 +4,21 @@
|
||||||
[](https://hub.docker.com/r/neodb/neodb)
|
[](https://hub.docker.com/r/neodb/neodb)
|
||||||
[](https://github.com/neodb-social/neodb/blob/main/LICENSE)
|
[](https://github.com/neodb-social/neodb/blob/main/LICENSE)
|
||||||
|
|
||||||
|
|
||||||
# 🧩 NeoDB
|
# 🧩 NeoDB
|
||||||
_mark the things you love._
|
_mark the things you love._
|
||||||
|
|
||||||
[NeoDB](https://neodb.net) (fka boofilsic) is an open source project and free service to help users manage, share and discover collections, reviews and ratings for culture products (e.g. books, movies, music, podcasts, games and performances) in Fediverse.
|
[NeoDB](https://neodb.net) (fka boofilsic) is an open source project and free service to help users manage, share and discover collections, reviews and ratings for culture products (e.g. books, movies, music, podcasts, games and performances) in Fediverse.
|
||||||
|
|
||||||
[NeoDB.social](https://neodb.social) is a free instance hosted by NeoDB developers, there are more [servers](https://neodb.net/servers/) and [apps](https://neodb.net/apps/) available. Your support is essential to keep them free and open-sourced.
|
[NeoDB.social](https://neodb.social) is a free instance hosted by NeoDB developers, there are more [servers](https://neodb.net/servers/) and [apps](https://neodb.net/apps/) available.
|
||||||
|
|
||||||
Follow us on [Fediverse](https://mastodon.online/@neodb), [Bluesky](https://bsky.app/profile/neodb.net) or join our [Discord community](https://discord.gg/QBHkrV8bxK) to share your ideas/questions/creations.
|
Follow us on [Fediverse](https://mastodon.online/@neodb), [Bluesky](https://bsky.app/profile/neodb.net) or join our [Discord community](https://discord.gg/QBHkrV8bxK) to share your ideas/questions/creations. Your support is essential to keep the services free and open-sourced.
|
||||||
|
|
||||||
[](https://mastodon.social/@neodb)
|
[](https://mastodon.social/@neodb)
|
||||||
[](https://discord.gg/QBHkrV8bxK)
|
[](https://discord.gg/QBHkrV8bxK)
|
||||||
[](https://ko-fi.com/neodb)
|
[](https://ko-fi.com/neodb)
|
||||||
|
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
- Manage a shared catalog of books/movies/tv shows/music album/games/podcasts/performances
|
- Manage a shared catalog of books/movies/tv shows/music album/games/podcasts/performances
|
||||||
+ search or create catalog items in each category
|
+ search or create catalog items in each category
|
||||||
|
|
@ -66,13 +68,17 @@ Follow us on [Fediverse](https://mastodon.online/@neodb), [Bluesky](https://bsky
|
||||||
- Other
|
- Other
|
||||||
+ i18n: English, Danish and Simp/Trad Chinese available; contribution for more languages welcomed
|
+ i18n: English, Danish and Simp/Trad Chinese available; contribution for more languages welcomed
|
||||||
|
|
||||||
|
|
||||||
## Host your own instance
|
## Host your own instance
|
||||||
Please see [docs/install.md](docs/install.md)
|
Please see [installation guide](https://neodb.net/install/).
|
||||||
|
|
||||||
|
|
||||||
## Contribution
|
## Contribution
|
||||||
- To build application with NeoDB API, documentation is available in [NeoDB API Developer Console](https://neodb.social/developer/)
|
- To build application with NeoDB API, documentation is available in [NeoDB API Developer Console](https://neodb.social/developer/)
|
||||||
- To help develop NeoDB, please see [docs/development.md](docs/development.md) for some basics to start with
|
- To help develop NeoDB, please see [development guide](https://neodb.net/development/) for some basics to start with
|
||||||
- To translate NeoDB to more languages, please join [our project on Weblate](https://hosted.weblate.org/projects/neodb/neodb/)
|
- To translate NeoDB to more languages, please join [our project on Weblate](https://hosted.weblate.org/projects/neodb/neodb/)
|
||||||
|
- Source code for [NeoDB documentation](https://neodb.net) can be found [here](https://github.com/neodb-social/neodb-doc)
|
||||||
|
|
||||||
|
|
||||||
## Sponsor
|
## Sponsor
|
||||||
If you like this project, please consider donating to [NeoDB.social on ko-fi](https://ko-fi.com/neodb), or [Takahē](https://www.patreon.com/takahe) and [NiceDB](https://patreon.com/tertius) without whom this project won't be possible.
|
If you like this project, please consider donating to [NeoDB.social on ko-fi](https://ko-fi.com/neodb), or [Takahē](https://www.patreon.com/takahe) and [NiceDB](https://patreon.com/tertius) without whom this project won't be possible.
|
||||||
|
|
|
||||||
|
|
@ -449,6 +449,7 @@ LANGUAGE_CODE, PREFERRED_LANGUAGES = _init_language_settings(
|
||||||
|
|
||||||
if TESTING: # force en if testing
|
if TESTING: # force en if testing
|
||||||
LANGUAGE_CODE = "en"
|
LANGUAGE_CODE = "en"
|
||||||
|
PREFERRED_LANGUAGES = ["en"]
|
||||||
|
|
||||||
LOCALE_PATHS = [os.path.join(BASE_DIR, "locale")]
|
LOCALE_PATHS = [os.path.join(BASE_DIR, "locale")]
|
||||||
|
|
||||||
|
|
@ -580,7 +581,7 @@ SEARCH_INDEX_NEW_ONLY = False
|
||||||
|
|
||||||
INDEX_ALIASES = env("INDEX_ALIASES")
|
INDEX_ALIASES = env("INDEX_ALIASES")
|
||||||
|
|
||||||
DOWNLOADER_SAVEDIR = env("NEODB_DOWNLOADER_SAVE_DIR", default="/tmp") # type: ignore
|
DOWNLOADER_SAVEDIR = env("NEODB_DOWNLOADER_SAVE_DIR", default="") # type: ignore
|
||||||
|
|
||||||
DISABLE_MODEL_SIGNAL = False # disable index and social feeds during importing/etc
|
DISABLE_MODEL_SIGNAL = False # disable index and social feeds during importing/etc
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -771,22 +771,22 @@ class Item(PolymorphicModel):
|
||||||
return not self.is_deleted and self.merged_to_item is None
|
return not self.is_deleted and self.merged_to_item is None
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def rating(self):
|
def rating_info(self):
|
||||||
from journal.models import Rating
|
from journal.models import Rating
|
||||||
|
|
||||||
return Rating.get_rating_for_item(self)
|
return Rating.get_info_for_item(self)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def rating(self):
|
||||||
|
return self.rating_info.get("average")
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def rating_count(self):
|
def rating_count(self):
|
||||||
from journal.models import Rating
|
return self.rating_info.get("count")
|
||||||
|
|
||||||
return Rating.get_rating_count_for_item(self)
|
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def rating_distribution(self):
|
def rating_distribution(self):
|
||||||
from journal.models import Rating
|
return self.rating_info.get("distribution")
|
||||||
|
|
||||||
return Rating.get_rating_distribution_for_item(self)
|
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def tags(self):
|
def tags(self):
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,8 @@ from catalog.music.utils import *
|
||||||
|
|
||||||
|
|
||||||
class BasicMusicTest(TestCase):
|
class BasicMusicTest(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_gtin(self):
|
def test_gtin(self):
|
||||||
self.assertIsNone(upc_to_gtin_13("018771208112X"))
|
self.assertIsNone(upc_to_gtin_13("018771208112X"))
|
||||||
self.assertIsNone(upc_to_gtin_13("999018771208112"))
|
self.assertIsNone(upc_to_gtin_13("999018771208112"))
|
||||||
|
|
@ -15,6 +17,8 @@ class BasicMusicTest(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class SpotifyTestCase(TestCase):
|
class SpotifyTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.Spotify_Album
|
t_id_type = IdType.Spotify_Album
|
||||||
t_id_value = "65KwtzkJXw7oT819NFWmEP"
|
t_id_value = "65KwtzkJXw7oT819NFWmEP"
|
||||||
|
|
@ -48,6 +52,8 @@ class SpotifyTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class DoubanMusicTestCase(TestCase):
|
class DoubanMusicTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.DoubanMusic
|
t_id_type = IdType.DoubanMusic
|
||||||
t_id_value = "33551231"
|
t_id_value = "33551231"
|
||||||
|
|
@ -74,6 +80,8 @@ class DoubanMusicTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class MultiMusicSitesTestCase(TestCase):
|
class MultiMusicSitesTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
@use_local_response
|
@use_local_response
|
||||||
def test_albums(self):
|
def test_albums(self):
|
||||||
url1 = "https://music.douban.com/subject/33551231/"
|
url1 = "https://music.douban.com/subject/33551231/"
|
||||||
|
|
@ -92,6 +100,8 @@ class MultiMusicSitesTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class BandcampTestCase(TestCase):
|
class BandcampTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.Bandcamp
|
t_id_type = IdType.Bandcamp
|
||||||
t_id_value = "intlanthem.bandcamp.com/album/in-these-times"
|
t_id_value = "intlanthem.bandcamp.com/album/in-these-times"
|
||||||
|
|
@ -119,6 +129,8 @@ class BandcampTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class DiscogsReleaseTestCase(TestCase):
|
class DiscogsReleaseTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.Discogs_Release
|
t_id_type = IdType.Discogs_Release
|
||||||
t_id_value = "25829341"
|
t_id_value = "25829341"
|
||||||
|
|
@ -155,6 +167,8 @@ class DiscogsReleaseTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class DiscogsMasterTestCase(TestCase):
|
class DiscogsMasterTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.Discogs_Master
|
t_id_type = IdType.Discogs_Master
|
||||||
t_id_value = "469004"
|
t_id_value = "469004"
|
||||||
|
|
@ -182,6 +196,8 @@ class DiscogsMasterTestCase(TestCase):
|
||||||
|
|
||||||
|
|
||||||
class AppleMusicTestCase(TestCase):
|
class AppleMusicTestCase(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
def test_parse(self):
|
def test_parse(self):
|
||||||
t_id_type = IdType.AppleMusic
|
t_id_type = IdType.AppleMusic
|
||||||
t_id_value = "1284391545"
|
t_id_value = "1284391545"
|
||||||
|
|
@ -201,8 +217,10 @@ class AppleMusicTestCase(TestCase):
|
||||||
self.assertEqual(site.ready, False)
|
self.assertEqual(site.ready, False)
|
||||||
site.get_resource_ready()
|
site.get_resource_ready()
|
||||||
self.assertEqual(site.ready, True)
|
self.assertEqual(site.ready, True)
|
||||||
self.assertEqual(site.resource.metadata["title"], "Kids Only")
|
self.assertEqual(
|
||||||
|
site.resource.metadata["localized_title"][0]["text"], "Kids Only"
|
||||||
|
)
|
||||||
self.assertEqual(site.resource.metadata["artist"], ["Leah Dou"])
|
self.assertEqual(site.resource.metadata["artist"], ["Leah Dou"])
|
||||||
self.assertIsInstance(site.resource.item, Album)
|
self.assertIsInstance(site.resource.item, Album)
|
||||||
self.assertEqual(site.resource.item.genre, ["Pop"])
|
self.assertEqual(site.resource.item.genre, ["Pop", "Music"])
|
||||||
self.assertEqual(site.resource.item.duration, 2371628)
|
self.assertEqual(site.resource.item.duration, 2368000)
|
||||||
|
|
|
||||||
|
|
@ -9,8 +9,9 @@ Scraping the website directly.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
from datetime import timedelta
|
||||||
|
|
||||||
import dateparser
|
from django.utils.dateparse import parse_duration
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from catalog.common import *
|
from catalog.common import *
|
||||||
|
|
@ -18,7 +19,6 @@ from catalog.models import *
|
||||||
from common.models.lang import (
|
from common.models.lang import (
|
||||||
SITE_DEFAULT_LANGUAGE,
|
SITE_DEFAULT_LANGUAGE,
|
||||||
SITE_PREFERRED_LANGUAGES,
|
SITE_PREFERRED_LANGUAGES,
|
||||||
detect_language,
|
|
||||||
)
|
)
|
||||||
from common.models.misc import uniq
|
from common.models.misc import uniq
|
||||||
|
|
||||||
|
|
@ -39,7 +39,6 @@ class AppleMusic(AbstractSite):
|
||||||
headers = {
|
headers = {
|
||||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0",
|
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0",
|
||||||
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
|
||||||
"Accept-Language": BasicDownloader.get_accept_language(),
|
|
||||||
"Accept-Encoding": "gzip, deflate",
|
"Accept-Encoding": "gzip, deflate",
|
||||||
"Connection": "keep-alive",
|
"Connection": "keep-alive",
|
||||||
"DNT": "1",
|
"DNT": "1",
|
||||||
|
|
@ -70,80 +69,63 @@ class AppleMusic(AbstractSite):
|
||||||
return locales
|
return locales
|
||||||
|
|
||||||
def scrape(self):
|
def scrape(self):
|
||||||
matched_content = None
|
matched_schema_data = None
|
||||||
localized_title = []
|
localized_title = []
|
||||||
localized_desc = []
|
localized_desc = []
|
||||||
for lang, locales in self.get_locales().items():
|
for lang, locales in self.get_locales().items():
|
||||||
for loc in locales: # waterfall thru all locales
|
for loc in locales: # waterfall thru all locales
|
||||||
url = f"https://music.apple.com/{loc}/album/{self.id_value}"
|
url = f"https://music.apple.com/{loc}/album/{self.id_value}"
|
||||||
try:
|
try:
|
||||||
|
tl = f"{lang}-{loc}" if lang == "zh" else lang
|
||||||
|
headers = {
|
||||||
|
"Accept-Language": tl,
|
||||||
|
}
|
||||||
|
headers.update(self.headers)
|
||||||
content = (
|
content = (
|
||||||
BasicDownloader(url, headers=self.headers).download().html()
|
BasicDownloader(url, headers=self.headers).download().html()
|
||||||
)
|
)
|
||||||
logger.info(f"got localized content from {url}")
|
logger.debug(f"got localized content from {url}")
|
||||||
elem = content.xpath(
|
txt: str = content.xpath(
|
||||||
"//script[@id='serialized-server-data']/text()"
|
"//script[@id='schema:music-album']/text()"
|
||||||
)
|
)[0] # type:ignore
|
||||||
txt: str = elem[0] # type:ignore
|
schema_data = json.loads(txt)
|
||||||
page_data = json.loads(txt)[0]
|
title = schema_data["name"]
|
||||||
album_data = page_data["data"]["sections"][0]["items"][0]
|
if title:
|
||||||
title = album_data["title"]
|
localized_title.append({"lang": tl, "text": title})
|
||||||
brief = album_data.get("modalPresentationDescriptor", {}).get(
|
try:
|
||||||
"paragraphText", ""
|
txt: str = content.xpath(
|
||||||
)
|
"//script[@id='serialized-server-data']/text()"
|
||||||
tl = detect_language(title + " " + brief)
|
)[0] # type:ignore
|
||||||
localized_title.append({"lang": tl, "text": title})
|
server_data = json.loads(txt)
|
||||||
if brief:
|
brief = server_data[0]["data"]["sections"][0]["items"][0][
|
||||||
localized_desc.append({"lang": tl, "text": brief})
|
"modalPresentationDescriptor"
|
||||||
if lang == SITE_DEFAULT_LANGUAGE or not matched_content:
|
]["paragraphText"]
|
||||||
matched_content = content
|
if brief:
|
||||||
|
localized_desc.append({"lang": tl, "text": brief})
|
||||||
|
except Exception:
|
||||||
|
server_data = brief = None
|
||||||
|
if lang == SITE_DEFAULT_LANGUAGE or not matched_schema_data:
|
||||||
|
matched_schema_data = schema_data
|
||||||
break
|
break
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
if matched_content is None:
|
if matched_schema_data is None: # no schema data found
|
||||||
raise ParseError(self, f"localized content for {self.url}")
|
raise ParseError(self, f"localized content for {self.url}")
|
||||||
elem = matched_content.xpath("//script[@id='serialized-server-data']/text()")
|
artist = [a["name"] for a in matched_schema_data.get("byArtist", [])]
|
||||||
txt: str = elem[0] # type:ignore
|
release_date = matched_schema_data.get("datePublished", None)
|
||||||
page_data = json.loads(txt)[0]
|
genre = matched_schema_data.get("genre", [])
|
||||||
album_data = page_data["data"]["sections"][0]["items"][0]
|
image_url = matched_schema_data.get("image", None)
|
||||||
title = album_data["title"]
|
track_list = [t["name"] for t in matched_schema_data.get("tracks", [])]
|
||||||
brief = album_data.get("modalPresentationDescriptor")
|
duration = round(
|
||||||
brief = brief.get("paragraphText") if brief else None
|
sum(
|
||||||
artist_list = album_data["subtitleLinks"]
|
(parse_duration(t["duration"]) or timedelta()).total_seconds() * 1000
|
||||||
artist = [item["title"] for item in artist_list]
|
for t in matched_schema_data.get("tracks", [])
|
||||||
|
)
|
||||||
track_data = page_data["data"]["seoData"]
|
|
||||||
date_elem = track_data.get("musicReleaseDate")
|
|
||||||
release_datetime = dateparser.parse(date_elem.strip()) if date_elem else None
|
|
||||||
release_date = (
|
|
||||||
release_datetime.strftime("%Y-%m-%d") if release_datetime else None
|
|
||||||
)
|
)
|
||||||
|
|
||||||
track_list = [
|
|
||||||
f"{i}. {track['attributes']['name']}"
|
|
||||||
for i, track in enumerate(track_data["ogSongs"], 1)
|
|
||||||
]
|
|
||||||
duration_list = [
|
|
||||||
track["attributes"].get("durationInMillis", 0)
|
|
||||||
for track in track_data["ogSongs"]
|
|
||||||
]
|
|
||||||
duration = int(sum(duration_list))
|
|
||||||
genre = track_data["schemaContent"].get("genre")
|
|
||||||
if genre:
|
|
||||||
genre = [
|
|
||||||
genre[0]
|
|
||||||
] # apple treat "Music" as a genre. Thus, only the first genre is obtained.
|
|
||||||
|
|
||||||
images = matched_content.xpath("//source[@type='image/jpeg']/@srcset")
|
|
||||||
image_elem: str = images[0] if images else "" # type:ignore
|
|
||||||
image_url = image_elem.split(" ")[0] if image_elem else None
|
|
||||||
|
|
||||||
pd = ResourceContent(
|
pd = ResourceContent(
|
||||||
metadata={
|
metadata={
|
||||||
"localized_title": uniq(localized_title),
|
"localized_title": uniq(localized_title),
|
||||||
"localized_description": uniq(localized_desc),
|
"localized_description": uniq(localized_desc),
|
||||||
"title": title,
|
|
||||||
"brief": brief,
|
|
||||||
"artist": artist,
|
"artist": artist,
|
||||||
"genre": genre,
|
"genre": genre,
|
||||||
"release_date": release_date,
|
"release_date": release_date,
|
||||||
|
|
|
||||||
|
|
@ -44,6 +44,7 @@ from catalog.common import (
|
||||||
)
|
)
|
||||||
from catalog.common.models import LANGUAGE_CHOICES_JSONFORM, LanguageListField
|
from catalog.common.models import LANGUAGE_CHOICES_JSONFORM, LanguageListField
|
||||||
from common.models.lang import RE_LOCALIZED_SEASON_NUMBERS, localize_number
|
from common.models.lang import RE_LOCALIZED_SEASON_NUMBERS, localize_number
|
||||||
|
from common.models.misc import uniq
|
||||||
|
|
||||||
|
|
||||||
class TVShowInSchema(ItemInSchema):
|
class TVShowInSchema(ItemInSchema):
|
||||||
|
|
@ -414,30 +415,35 @@ class TVSeason(Item):
|
||||||
- "Show Title Season X" with some localization
|
- "Show Title Season X" with some localization
|
||||||
"""
|
"""
|
||||||
s = super().display_title
|
s = super().display_title
|
||||||
if self.parent_item and (
|
if self.parent_item:
|
||||||
RE_LOCALIZED_SEASON_NUMBERS.sub("", s) == ""
|
if (
|
||||||
or s == self.parent_item.display_title
|
RE_LOCALIZED_SEASON_NUMBERS.sub("", s) == ""
|
||||||
):
|
or s == self.parent_item.display_title
|
||||||
if self.parent_item.get_season_count() == 1:
|
):
|
||||||
return self.parent_item.display_title
|
if self.parent_item.get_season_count() == 1:
|
||||||
elif self.season_number:
|
return self.parent_item.display_title
|
||||||
return _("{show_title} Season {season_number}").format(
|
elif self.season_number:
|
||||||
show_title=self.parent_item.display_title,
|
return _("{show_title} Season {season_number}").format(
|
||||||
season_number=localize_number(self.season_number),
|
show_title=self.parent_item.display_title,
|
||||||
)
|
season_number=localize_number(self.season_number),
|
||||||
else:
|
)
|
||||||
return f"{self.parent_item.display_title} {s}"
|
else:
|
||||||
|
return f"{self.parent_item.display_title} {s}"
|
||||||
|
elif self.parent_item.display_title not in s:
|
||||||
|
return f"{self.parent_item.display_title} ({s})"
|
||||||
return s
|
return s
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def additional_title(self) -> list[str]:
|
def additional_title(self) -> list[str]:
|
||||||
title = self.display_title
|
title = self.display_title
|
||||||
return [
|
return uniq(
|
||||||
t["text"]
|
[
|
||||||
for t in self.localized_title
|
t["text"]
|
||||||
if t["text"] != title
|
for t in self.localized_title
|
||||||
and RE_LOCALIZED_SEASON_NUMBERS.sub("", t["text"]) != ""
|
if t["text"] != title
|
||||||
]
|
and RE_LOCALIZED_SEASON_NUMBERS.sub("", t["text"]) != ""
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
def to_indexable_titles(self) -> list[str]:
|
def to_indexable_titles(self) -> list[str]:
|
||||||
titles = [t["text"] for t in self.localized_title if t["text"]]
|
titles = [t["text"] for t in self.localized_title if t["text"]]
|
||||||
|
|
|
||||||
|
|
@ -83,7 +83,7 @@ def nodeinfo2(request):
|
||||||
|
|
||||||
def _error_response(request, status: int, exception=None, default_message=""):
|
def _error_response(request, status: int, exception=None, default_message=""):
|
||||||
message = str(exception) if exception else default_message
|
message = str(exception) if exception else default_message
|
||||||
if request.headers.get("HTTP_ACCEPT").endswith("json"):
|
if request.headers.get("HTTP_ACCEPT", "").endswith("json"):
|
||||||
return JsonResponse({"error": message}, status=status)
|
return JsonResponse({"error": message}, status=status)
|
||||||
if (
|
if (
|
||||||
request.headers.get("HTTP_HX_REQUEST") is not None
|
request.headers.get("HTTP_HX_REQUEST") is not None
|
||||||
|
|
|
||||||
81
docs/api.md
81
docs/api.md
|
|
@ -1,81 +0,0 @@
|
||||||
# API
|
|
||||||
|
|
||||||
## Endpoints
|
|
||||||
|
|
||||||
NeoDB has a set of API endpoints mapping to its functions like marking a book or listing collections, they can be found in swagger based API documentation at `/developer/` of your running instance, [a version of it](https://neodb.social/developer/) is available on our flagship instance.
|
|
||||||
|
|
||||||
NeoDB also supports a subset of Mastodon API, details can be found in [Mastodon API documentation](https://docs.joinmastodon.org/api/).
|
|
||||||
|
|
||||||
Both set of APIs can be accessed by the same access token.
|
|
||||||
|
|
||||||
## How to authorize
|
|
||||||
|
|
||||||
### Create an application
|
|
||||||
|
|
||||||
you must have at least one URL included in the Redirect URIs field, e.g. `https://example.org/callback`, or use `urn:ietf:wg:oauth:2.0:oob` if you don't have a callback URL.
|
|
||||||
|
|
||||||
```
|
|
||||||
curl https://neodb.social/api/v1/apps \
|
|
||||||
-d client_name=MyApp \
|
|
||||||
-d redirect_uris=https://example.org/callback \
|
|
||||||
-d website=https://my.site
|
|
||||||
```
|
|
||||||
|
|
||||||
and save of the `client_id` and `client_secret` returned in the response:
|
|
||||||
|
|
||||||
```
|
|
||||||
{
|
|
||||||
"client_id": "CLIENT_ID",
|
|
||||||
"client_secret": "CLIENT_SECRET",
|
|
||||||
"name": "MyApp",
|
|
||||||
"redirect_uri": "https://example.org/callback",
|
|
||||||
"vapid_key": "PUSH_KEY",
|
|
||||||
"website": "https://my.site"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### Guide your user to open this URL
|
|
||||||
|
|
||||||
```
|
|
||||||
https://neodb.social/oauth/authorize?response_type=code&client_id=CLIENT_ID&redirect_uri=https://example.org/callback&scope=read+write
|
|
||||||
```
|
|
||||||
|
|
||||||
### Once authorizated by user, it will redirect to `https://example.org/callback` with a `code` parameter:
|
|
||||||
|
|
||||||
```
|
|
||||||
https://example.org/callback?code=AUTH_CODE
|
|
||||||
```
|
|
||||||
|
|
||||||
### Obtain access token with the following POST request:
|
|
||||||
|
|
||||||
```
|
|
||||||
curl https://neodb.social/oauth/token \
|
|
||||||
-d "client_id=CLIENT_ID" \
|
|
||||||
-d "client_secret=CLIENT_SECRET" \
|
|
||||||
-d "code=AUTH_CODE" \
|
|
||||||
-d "redirect_uri=https://example.org/callback" \
|
|
||||||
-d "grant_type=authorization_code"
|
|
||||||
```
|
|
||||||
|
|
||||||
and access token will be returned in the response:
|
|
||||||
|
|
||||||
```
|
|
||||||
{
|
|
||||||
"access_token": "ACCESS_TOKEN",
|
|
||||||
"token_type": "Bearer",
|
|
||||||
"scope": "read write"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Use the access token to access protected endpoints like `/api/me`
|
|
||||||
|
|
||||||
```
|
|
||||||
curl -H "Authorization: Bearer ACCESS_TOKEN" -X GET https://neodb.social/api/me
|
|
||||||
```
|
|
||||||
|
|
||||||
and response will be returned accordingly:
|
|
||||||
|
|
||||||
```
|
|
||||||
{"url": "https://neodb.social/users/xxx/", "external_acct": "xxx@yyy.zzz", "display_name": "XYZ", "avatar": "https://yyy.zzz/xxx.gif"}
|
|
||||||
```
|
|
||||||
19
docs/apps.md
19
docs/apps.md
|
|
@ -1,19 +0,0 @@
|
||||||
# Apps
|
|
||||||
|
|
||||||
NeoDB web version will provide the most features and experience, while some third-party apps are also available below.
|
|
||||||
|
|
||||||
## Apps for NeoDB
|
|
||||||
|
|
||||||
A few apps for NeoDB are being actively developed:
|
|
||||||
|
|
||||||
- [Piecelet](https://piecelet.app) by `@piecelet@mastodon.social` - [App Store](https://apps.apple.com/app/piecelet-for-neodb/id6739444863) / [Source Code](https://github.com/lcandy2/neodb-app)
|
|
||||||
- [Chihu](https://chihu.app) by `@chihu@mastodon.social` - [Test Flight](https://testflight.apple.com/join/WmbnP9Vx)
|
|
||||||
|
|
||||||
These apps are not affiliated with NeoDB, but they are being developed with the support of this community. If you are also developing an app for NeoDB, and wish to share that with the community, please [edit this file](https://github.com/neodb-social/neodb/edit/main/docs/apps.md) and submit a pull request.
|
|
||||||
|
|
||||||
|
|
||||||
## Mastodon apps
|
|
||||||
|
|
||||||
[Mastodon compatible mobile and native apps](https://joinmastodon.org/apps) can be used to log in and utilize the micro-blogging features in NeoDB server.
|
|
||||||
|
|
||||||
In addition to micro-blogging, Mastodon compatible can also be used to take note on book you are currently reading. Just head to bookmark section in your app, your currently reading books are listed there as bookmarked posts, replying any of them will make a note for that book.
|
|
||||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 42 KiB |
|
|
@ -1,210 +0,0 @@
|
||||||
# Configuration
|
|
||||||
|
|
||||||
|
|
||||||
## Important settings you may want to change first
|
|
||||||
|
|
||||||
absolutely set these in `.env` before start the instance for the first time:
|
|
||||||
|
|
||||||
- `NEODB_SECRET_KEY` - 50 characters of random string, no white space
|
|
||||||
- `NEODB_SITE_NAME` - the name of your site
|
|
||||||
- `NEODB_SITE_DOMAIN` - the domain name of your site
|
|
||||||
|
|
||||||
**`NEODB_SECRET_KEY` and `NEODB_SITE_DOMAIN` must not be changed later.**
|
|
||||||
|
|
||||||
if you are doing debug or development:
|
|
||||||
|
|
||||||
- `NEODB_DEBUG` - True will turn on debug for both neodb and takahe, turn off relay, and reveal self as debug mode in nodeinfo (so peers won't try to run fedi search on this node)
|
|
||||||
- `NEODB_IMAGE` - the docker image to use, `neodb/neodb:edge` for the main branch
|
|
||||||
|
|
||||||
## Settings for customization
|
|
||||||
|
|
||||||
- `NEODB_SITE_LOGO`
|
|
||||||
- `NEODB_SITE_ICON`
|
|
||||||
- `NEODB_USER_ICON`
|
|
||||||
- `NEODB_SITE_COLOR` - one of [these color schemes](https://picocss.com/docs/colors)
|
|
||||||
- `NEODB_SITE_INTRO`
|
|
||||||
- `NEODB_SITE_HEAD`
|
|
||||||
- `NEODB_SITE_DESCRIPTION`
|
|
||||||
- `NEODB_PREFERRED_LANGUAGES` - preferred languages when importing titles from 3rd party sites like TMDB and Steam, comma-separated list of ISO-639-1 two-letter codes, `en,zh` by default. It can includes languages with no UI translations yet, e.g. if set to `ja,en,zh`, NeoDB scraper will fetch catalog metadata in three languages if they are available from third party sites, Japanese users (= whose browser language set to ja-JP) will see English UI with Japanese metadata.
|
|
||||||
- `NEODB_DISCOVER_FILTER_LANGUAGE` - `False` by default; when set to `True`, `/discover/` will only show items with languages match one of `NEODB_PREFERRED_LANGUAGES`.
|
|
||||||
- `NEODB_DISCOVER_SHOW_LOCAL_ONLY` - `False` by default; when set to `True`, only show items marked by local users rather than entire network on `/discover/`
|
|
||||||
- `NEODB_DISCOVER_UPDATE_INTERVAL` - minutes between each update for popular items on `/discover/`
|
|
||||||
- `NEODB_SITE_LINKS` - a list of title and links to show in the footer, comma separated, e.g. `Feedback=https://discord.gg/8KweCuApaK,ToS=/pages/rules/`
|
|
||||||
- `NEODB_INVITE_ONLY` - `False` by default, set to `True` to require invite code(generated by `neodb-manage invite --create`) to register
|
|
||||||
- `NEODB_ENABLE_LOCAL_ONLY` - `False` by default, set to `True` to allow user to post marks as "local public"
|
|
||||||
- `NEODB_LOGIN_MASTODON_WHITELIST` - a list of Mastodon instances to allow login from, comma separated
|
|
||||||
- `NEODB_EMAIL_FROM` - the email address to send email from
|
|
||||||
- `NEODB_EMAIL_URL` - email sender configuration, e.g.
|
|
||||||
- `smtp://<username>:<password>@<host>:<port>`
|
|
||||||
- `smtp+tls://<username>:<password>@<host>:<port>`
|
|
||||||
- `smtp+ssl://<username>:<password>@<host>:<port>`
|
|
||||||
- `anymail://<anymail_backend_name>?<anymail_args>`, to send email via email service providers, see [anymail doc](https://anymail.dev/)
|
|
||||||
|
|
||||||
## Settings for administration
|
|
||||||
- `DISCORD_WEBHOOKS` - Discord channel to send notification about user submitted suggestion and changes, e.g. `suggest=https://discord.com/api/webhooks/123/abc,audit=https://discord.com/api/webhooks/123/def`. Both suggest and audit channels must be in forum mode.
|
|
||||||
- `NEODB_SENTRY_DSN` , `TAKAHE_SENTRY_DSN` - [Sentry](https://sentry.io/) DSN to log errors.
|
|
||||||
|
|
||||||
## Settings for Federation
|
|
||||||
|
|
||||||
- `NEODB_SEARCH_PEERS` is empty by default, which means NeoDB will search all known peers running production version of NeoDB when user look for items. This can be set to a comma-separated list of host names, so that NeoDB will only search those servers; or search no other peers if set to just `-`.
|
|
||||||
|
|
||||||
- `NEODB_DISABLE_DEFAULT_RELAY` is set to `False` by default, the server will send and receive public posts from `relay.neodb.net`.
|
|
||||||
|
|
||||||
`relay.neodb.net` is [open sourced](https://github.com/neodb-social/neodb-relay) and operated by NeoDB developers, it works like most ActivityPub relays except it only relays between NeoDB instances, it helps public information like catalogs and trends flow between NeoDB instances. You may set it to `True` if you don't want to relay public posts with other NeoDB instances.
|
|
||||||
|
|
||||||
|
|
||||||
## Settings for external item sources
|
|
||||||
|
|
||||||
- `SPOTIFY_API_KEY` - base64('CLIENT_ID:SECRET'), see [spotify doc](https://developer.spotify.com/documentation/web-api/tutorials/client-credentials-flow)
|
|
||||||
- `TMDB_API_V3_KEY` - API v3 key from [TMDB](https://developer.themoviedb.org/)
|
|
||||||
- `GOOGLE_API_KEY` - API key for [Google Books](https://developers.google.com/books/docs/v1/using)
|
|
||||||
- `DISCOGS_API_KEY` - personal access token from [Discogs](https://www.discogs.com/settings/developers)
|
|
||||||
- `IGDB_API_CLIENT_ID`, `IGDB_API_CLIENT_SECRET` - IGDB [keys](https://api-docs.igdb.com/)
|
|
||||||
- `NEODB_SEARCH_SITES` is empty by default, which means NeoDB will search all available sources. This can be set to a comma-separated list of site names (e.g. `goodreads,googlebooks,spotify,tmdb,igdb,bandcamp,apple_podcast`), so that NeoDB will only search those sites; or not search any of them if set to just `-`.
|
|
||||||
|
|
||||||
|
|
||||||
## Other maintenance tasks
|
|
||||||
|
|
||||||
Add alias to your shell for easier access
|
|
||||||
|
|
||||||
```
|
|
||||||
alias neodb-manage='docker-compose --profile production run --rm shell neodb-manage'
|
|
||||||
```
|
|
||||||
|
|
||||||
Toggle user's active, staff and super user status
|
|
||||||
|
|
||||||
```
|
|
||||||
neodb-manage user --active <username>
|
|
||||||
neodb-manage user --staff <username>
|
|
||||||
neodb-manage user --super <username>
|
|
||||||
```
|
|
||||||
|
|
||||||
create a super user; delete a user / remote identity (`takahe-stator` and `neodb-worker` containers must be running to complete the deletion)
|
|
||||||
```
|
|
||||||
neodb-manage createsuperuser
|
|
||||||
neodb-manage user --delete username
|
|
||||||
neodb-manage user --delete username@remote.instance
|
|
||||||
```
|
|
||||||
|
|
||||||
Create an invite link
|
|
||||||
|
|
||||||
```
|
|
||||||
neodb-manage invite --create
|
|
||||||
```
|
|
||||||
|
|
||||||
Manage user tasks and cron jobs
|
|
||||||
|
|
||||||
```
|
|
||||||
neodb-manage task --list
|
|
||||||
neodb-manage cron --list
|
|
||||||
```
|
|
||||||
|
|
||||||
Manage search index
|
|
||||||
```
|
|
||||||
neo-manage index --reindex
|
|
||||||
```
|
|
||||||
|
|
||||||
Crawl links
|
|
||||||
```
|
|
||||||
neodb-manage cat [--save] <url> # parse / save a supported link
|
|
||||||
neodb-manage crawl <url> # crawl all recognizable links from a page
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Run PostgresQL/Redis/Typesense without Docker
|
|
||||||
|
|
||||||
It's currently possible but quite cumbersome to run without Docker, hence not recommended. However it's possible to only use docker to run neodb server but reuse existing PostgresQL/Redis/Typesense servers with `compose.override.yml`, an example for reference:
|
|
||||||
|
|
||||||
```
|
|
||||||
services:
|
|
||||||
redis:
|
|
||||||
profiles: ['disabled']
|
|
||||||
typesense:
|
|
||||||
profiles: ['disabled']
|
|
||||||
neodb-db:
|
|
||||||
profiles: ['disabled']
|
|
||||||
takahe-db:
|
|
||||||
profiles: ['disabled']
|
|
||||||
migration:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
neodb-web:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
healthcheck: !reset {}
|
|
||||||
neodb-web-api:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
healthcheck: !reset {}
|
|
||||||
neodb-worker:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
neodb-worker-extra:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
takahe-web:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
takahe-stator:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
shell:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
root:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-neodb-web:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-neodb-worker:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-takahe-web:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-takahe-stator:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-shell:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
dev-root:
|
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
depends_on: !reset []
|
|
||||||
```
|
|
||||||
(`extra_hosts` is only needed if PostgresQL/Redis/Typesense is on your host server)
|
|
||||||
|
|
||||||
|
|
||||||
## Multiple instances on one server
|
|
||||||
|
|
||||||
It's possible to run multiple clusters in one host server with docker compose, as long as `NEODB_SITE_DOMAIN`, `NEODB_PORT` and `NEODB_DATA` are different.
|
|
||||||
|
|
||||||
|
|
||||||
## Scaling up
|
|
||||||
|
|
||||||
For high-traffic instance, spin up these configurations to a higher number, as long as the host server can handle them:
|
|
||||||
|
|
||||||
- `NEODB_WEB_WORKER_NUM`
|
|
||||||
- `NEODB_API_WORKER_NUM`
|
|
||||||
- `NEODB_RQ_WORKER_NUM`
|
|
||||||
- `TAKAHE_WEB_WORKER_NUM`
|
|
||||||
- `TAKAHE_STATOR_CONCURRENCY`
|
|
||||||
- `TAKAHE_STATOR_CONCURRENCY_PER_MODEL`
|
|
||||||
|
|
||||||
Further scaling up with multiple nodes (e.g. via Kubernetes) is beyond the scope of this document, but consider run db/redis/typesense separately, and then duplicate web/worker/stator containers as long as connections and mounts are properly configured; `migration` only runs once when start or upgrade, it should be kept that way.
|
|
||||||
|
|
@ -1,187 +0,0 @@
|
||||||
Development
|
|
||||||
===========
|
|
||||||
|
|
||||||
Overview
|
|
||||||
--------
|
|
||||||
NeoDB is a Django project, and it runs side by side with a [modified version](https://github.com/neodb-social/neodb-takahe) of [Takahē](https://github.com/jointakahe/takahe) (a separate Django project, code in `neodb_takahe` folder of this repo as submodule). They communicate with each other mainly thru database and task queue, [the diagram](troubleshooting.md#containers) demonstrates a typical architecture. Currently the two are loosely coupled, so you may take either one offline without immediate impact on the other, which makes it very easy to conduct maintenance and troubleshooting separately. In the future, they may get combined but it's not decided and will not be decided very soon.
|
|
||||||
|
|
||||||
|
|
||||||
Prerequisite
|
|
||||||
------------
|
|
||||||
- Python 3.12.x
|
|
||||||
- Docker Compose v2 or newer
|
|
||||||
|
|
||||||
|
|
||||||
Prepare the code
|
|
||||||
----------------
|
|
||||||
When checking out NeoDB source code, make sure submodules are also checked out:
|
|
||||||
```
|
|
||||||
git clone https://github.com/neodb-social/neodb.git
|
|
||||||
cd neodb
|
|
||||||
git submodule update --init
|
|
||||||
```
|
|
||||||
|
|
||||||
Install [rye](http://rye.astral.sh) package manager, packages and pre-commit hooks:
|
|
||||||
```
|
|
||||||
curl -sSf https://rye.astral.sh/get | bash
|
|
||||||
rye sync
|
|
||||||
. .venv/bin/activate
|
|
||||||
pre-commit install
|
|
||||||
```
|
|
||||||
|
|
||||||
To develop Takahe, install requirements(-dev) and pre-commit hooks for `neodb-takahe` project as well, preferably using a different virtual environment.
|
|
||||||
|
|
||||||
Note: the virtual environments and packages installed in this step are mostly for linting, the actual virtual environments and packages are from NeoDB docker image, and they can be configured differently, more on this later in this document.
|
|
||||||
|
|
||||||
|
|
||||||
Start local instance for development
|
|
||||||
------------------------------------
|
|
||||||
Follow [install guide](install.md) to create `.env` in the root folder of NeoDB code, including at least these configuration:
|
|
||||||
```
|
|
||||||
NEODB_SITE_NAME="My Test"
|
|
||||||
NEODB_SITE_DOMAIN=mydomain.dev
|
|
||||||
NEODB_SECRET_KEY=_random_string__50_characters_of_length__no_whitespaces_
|
|
||||||
NEODB_IMAGE=neodb/neodb:edge
|
|
||||||
NEODB_DEBUG=True
|
|
||||||
```
|
|
||||||
|
|
||||||
Download docker images and start pgsql/redis/typesense before initializing database schema:
|
|
||||||
```
|
|
||||||
docker compose --profile dev pull
|
|
||||||
docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Initialize database schema:
|
|
||||||
```
|
|
||||||
docker compose --profile dev run --rm dev-shell takahe-manage collectstatic --noinput
|
|
||||||
docker compose --profile dev run --rm dev-shell neodb-init
|
|
||||||
```
|
|
||||||
|
|
||||||
Start the cluster:
|
|
||||||
```
|
|
||||||
docker compose --profile dev up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Watch the logs:
|
|
||||||
```
|
|
||||||
docker compose --profile dev logs -f
|
|
||||||
```
|
|
||||||
|
|
||||||
Now the local development instance is ready to serve at `http://localhost:8000`, but to develop or test anything related with ActivityPub, reverse proxying it from externally reachable https://`NEODB_SITE_DOMAIN`/ is required; https is optional theoretically but in reality required for various compatibility reasons.
|
|
||||||
|
|
||||||
Note: `dev` profile is for development only, and quite different from `production`, so always use `--profile dev` instead of `--profile production`, more on those differences later in this document.
|
|
||||||
|
|
||||||
|
|
||||||
Common development tasks
|
|
||||||
------------------------
|
|
||||||
Shutdown the cluster:
|
|
||||||
```
|
|
||||||
docker compose --profile dev down
|
|
||||||
```
|
|
||||||
|
|
||||||
Restart background tasks (unlike web servers, background tasks don't auto reload after code change):
|
|
||||||
```
|
|
||||||
docker-compose --profile dev restart dev-neodb-worker dev-takahe-stator
|
|
||||||
```
|
|
||||||
|
|
||||||
When updating code, always update submodules:
|
|
||||||
```
|
|
||||||
git pull
|
|
||||||
git submodule update --init
|
|
||||||
```
|
|
||||||
With newer git 2.15+, you main use `git pull --recurse-submodules` or `git config --global submodule.recurse true` to make it automatic.
|
|
||||||
|
|
||||||
|
|
||||||
To save some typing, consider adding some aliases to `~/.profile`:
|
|
||||||
```
|
|
||||||
alias neodb-logs='docker compose --profile dev logs'
|
|
||||||
alias neodb-shell='docker compose --profile dev run --rm dev-shell'
|
|
||||||
alias neodb-manage='docker compose --profile dev run --rm dev-shell neodb-manage'
|
|
||||||
```
|
|
||||||
|
|
||||||
Always use `neodb-init`, not `python3 manage.py migrate`, to update db schema after updating code:
|
|
||||||
```
|
|
||||||
neodb-shell neodb-init
|
|
||||||
```
|
|
||||||
|
|
||||||
Run unit test:
|
|
||||||
```
|
|
||||||
neodb-manage test
|
|
||||||
```
|
|
||||||
|
|
||||||
Update translations:
|
|
||||||
```
|
|
||||||
django-admin makemessages --no-wrap --no-obsolete -i .venv -i neodb-takahe --keep-pot -l zh_Hans -l zh_Hant
|
|
||||||
|
|
||||||
# edit .po files, run the following to make sure edits are correct
|
|
||||||
|
|
||||||
django-admin makemessages --no-wrap --no-obsolete -i .venv -i neodb-takahe --keep-pot -l zh_Hans -l zh_Hant
|
|
||||||
django-admin compilemessages -i .venv -i neodb-takahe
|
|
||||||
```
|
|
||||||
|
|
||||||
Preview documentation:
|
|
||||||
```
|
|
||||||
python -m mkdocs serve
|
|
||||||
```
|
|
||||||
|
|
||||||
Development in Docker Compose
|
|
||||||
-----------------------------
|
|
||||||
The `dev` profile is different from `production`:
|
|
||||||
|
|
||||||
- code in `NEODB_SRC` (default: .) and `TAKAHE_SRC` (default: ./neodb-takahe) will be mounted and used in the container instead of code in the image
|
|
||||||
- `runserver` with autoreload will be used instead of `gunicorn` for both neodb and takahe web server
|
|
||||||
- /static/ and /s/ url are not map to pre-generated/collected static file path, `NEODB_DEBUG=True` is required locate static files from source code
|
|
||||||
- one `rqworker` container will be started, instead of two
|
|
||||||
- use `dev-shell` and `dev-root` to invoke shells, instead of `shell` and `root`
|
|
||||||
- there's no automatic `migration` container, but it can be triggered manually via `docker compose run dev-shell neodb-init`
|
|
||||||
|
|
||||||
Note:
|
|
||||||
|
|
||||||
- Python virtual environments inside docker image, which are `/neodb-venv` and `/takahe-venv`, will be used by default. They can be changed to different locations with `TAKAHE_VENV` and `NEODB_VENV` if needed, usually in a case of development code using a package not in docker venv.
|
|
||||||
- Some packages inside python virtual environments are platform dependent, so mount venv built by macOS host into the Linux container will likely not work.
|
|
||||||
- Python servers are launched as `app` user, who has no write access to anywhere except /tmp and media path, that's by design.
|
|
||||||
- Database/redis/typesense used in the container cluster are not accessible from host directly, which is by design. Querying them can be done by one of the following:
|
|
||||||
- `neodb-manage dbshell`
|
|
||||||
- `neodb-shell redis-cli -h redis`
|
|
||||||
- or create `compose.override.yml` to uncomment `ports` section.
|
|
||||||
- To expose the neodb and takahe web server directly, in the folder for configuration, create `compose.override.yml` with the following content:
|
|
||||||
|
|
||||||
```
|
|
||||||
services:
|
|
||||||
dev-neodb-web:
|
|
||||||
ports:
|
|
||||||
- "8001:8000"
|
|
||||||
|
|
||||||
dev-takahe-web:
|
|
||||||
ports:
|
|
||||||
- "8002:8000"
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
Development with Github Codespace
|
|
||||||
---------------------------------
|
|
||||||
At the time of writing, docker compose will work in Github Codespace by adding this in `.env`:
|
|
||||||
|
|
||||||
```
|
|
||||||
NEODB_SITE_DOMAIN=${CODESPACE_NAME}-8000.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
Applications
|
|
||||||
------------
|
|
||||||
Main Django apps for NeoDB:
|
|
||||||
|
|
||||||
- `users` manages user in typical Django fashion
|
|
||||||
- `mastodon` this leverages [Mastodon API](https://docs.joinmastodon.org/client/intro/), [Threads API](https://developers.facebook.com/docs/threads/) and [ATProto}(https://atproto.com) for user login and data sync
|
|
||||||
- `catalog` manages different types of items user may collect, and scrapers to fetch from external resources, see [catalog.md](internals/catalog.md) for more details
|
|
||||||
- `journal` manages user created content(review/ratings) and lists(collection/shelf/tag/note), see [journal.md](internals/journal.md) for more details
|
|
||||||
- `social` present timeline and notification for local users
|
|
||||||
- `takahe` communicate with Takahe (a separate Django server, run side by side with this server, code in `neodb_takahe` as submodule), see [federation.md](internals/federation.md) for customization of ActivityPub protocol
|
|
||||||
- `legacy` this is only used by instances upgraded from 0.4.x and earlier, to provide a link mapping from old urls to new ones. If your journey starts with 0.5 and later, feel free to ignore it.
|
|
||||||
|
|
||||||
|
|
||||||
Miscellaneous notes
|
|
||||||
-------------------
|
|
||||||
If models in `takahe/models.py` are changed, instead of adding incremental migrations, just regenerate `takahe/migrations/0001_initial.py` instead, because these migrations will never be applied except for constructing a test database.
|
|
||||||
|
|
||||||
A `libsass` wheel is stored in the repo to speed up docker image building process in Github Action.
|
|
||||||
|
|
@ -1,69 +0,0 @@
|
||||||
# Features
|
|
||||||
|
|
||||||
NeoDB has [various features](features.md), and you may image it as a mix of Mastodon, Goodreads, Letterboxd, RateYourMusic, Podchaser, and more.
|
|
||||||
|
|
||||||
## Public catalog
|
|
||||||
|
|
||||||
- a shared catalog of books/movies/tv shows/music album/games/podcasts/performances
|
|
||||||
- search or create catalog items in each category
|
|
||||||
- one click create item with links to 3rd party sites:
|
|
||||||
- Goodreads
|
|
||||||
- IMDB
|
|
||||||
- The Movie Database
|
|
||||||
- Douban
|
|
||||||
- Google Books
|
|
||||||
- Discogs
|
|
||||||
- Spotify
|
|
||||||
- Apple Music
|
|
||||||
- Bandcamp
|
|
||||||
- Steam
|
|
||||||
- IGDB
|
|
||||||
- Bangumi
|
|
||||||
- Board Game Geek
|
|
||||||
- any RSS link to a podcast
|
|
||||||
- ...[full list](sites.md)
|
|
||||||
|
|
||||||
|
|
||||||
## Personal collections
|
|
||||||
|
|
||||||
- mark an item as wishlist/in progress/complete/dropped
|
|
||||||
- rate and write reviews for an item
|
|
||||||
- write notes for an item with progress (e.g. reading notes at page 42)
|
|
||||||
- create tags for an item, either privately or publicly
|
|
||||||
- create and share list of items
|
|
||||||
- tracking progress of a list (e.g. personal reading challenges)
|
|
||||||
- Import and export full user data archive
|
|
||||||
- import list or archives from some 3rd party sites:
|
|
||||||
- Goodreads reading list
|
|
||||||
- Letterboxd watch list
|
|
||||||
- Douban archive (via [Doufen](https://doufen.org/))
|
|
||||||
|
|
||||||
|
|
||||||
## Social
|
|
||||||
|
|
||||||
- view home feed with friends' activities
|
|
||||||
- every activity can be set as viewable to self/follower-only/public
|
|
||||||
- eligible items, e.g. podcasts and albums, are playable in feed
|
|
||||||
- login with other Fediverse identity and import social graph
|
|
||||||
- supported servers: Mastodon/Pleroma/Firefish/GoToSocial/Pixelfed/friendica/Takahē
|
|
||||||
- login with Bluesky / ATProto identity and import social graph
|
|
||||||
- login with threads.net (requires app verification by Meta)
|
|
||||||
- share collections and reviews to Fediverse/Bluesky/Threads
|
|
||||||
- ActivityPub support
|
|
||||||
- NeoDB users can follow and interact with users on other ActivityPub services like Mastodon and Pleroma
|
|
||||||
- NeoDB instances communicate with each other via an extended version of ActivityPub
|
|
||||||
- NeoDB instances may share public rating and reviews with a default relay
|
|
||||||
- implementation is based on [Takahē](https://jointakahe.org/) server
|
|
||||||
|
|
||||||
|
|
||||||
## API
|
|
||||||
- Mastodon compatible API
|
|
||||||
- most mastodon compatible apps are compatible with NeoDB
|
|
||||||
- NeoDB API to manage reviews and collections
|
|
||||||
|
|
||||||
## Languages
|
|
||||||
|
|
||||||
- English
|
|
||||||
- Simplified Chinese
|
|
||||||
- Traditional Chinese
|
|
||||||
- more to come and your contributions are welcomed!
|
|
||||||
|
|
@ -1,47 +0,0 @@
|
||||||
NeoDB is an open-source software and global community platform [since 2021](origin.md). It helps users to manage and explore collections, reviews, and ratings for various cultural products, including books, movies, music, podcasts, games, and performances. Additionally, users can share their collections, publish microblogs, and engage with others in the Fediverse. NeoDB integrates the functionalities of platforms like Goodreads, Letterboxd, RateYourMusic, and Podchaser, among others. It also supports self-hosting and interconnection through containerized deployment and the ActivityPub protocol.
|
|
||||||
|
|
||||||
This project is licensed under AGPLv3.
|
|
||||||
|
|
||||||
|
|
||||||
## How to use
|
|
||||||
|
|
||||||
Please [find a server](servers.md) to register with your email, Mastodon or Bluesky ID.
|
|
||||||
|
|
||||||
Besides the web version of your chosen server, you may also use [various apps](apps.md) to login and manage your collections.
|
|
||||||
|
|
||||||
|
|
||||||
## Host your own instance
|
|
||||||
|
|
||||||
You can [install and run your own instance of NeoDB](install.md). If you decide to share your server with the public, please inform us so that we can add it to the list above.
|
|
||||||
|
|
||||||
All instances interact with each other in the Fediverse via ActivityPub, allowing you to follow users from other NeoDB instances or instances running other ActivityPub software, such as Mastodon.
|
|
||||||
|
|
||||||
|
|
||||||
## API, Development and Translation
|
|
||||||
|
|
||||||
- NeoDB offers [APIs to manage user collections](api.md), and [Mastodon client compatible API](https://docs.joinmastodon.org/client/) to manage user posts.
|
|
||||||
- For those interested in developing for NeoDB, please refer to the [development](development.md) section for basic instructions to get started.
|
|
||||||
- To help translate NeoDB to more languages, please join [our project on Weblate](https://hosted.weblate.org/projects/neodb/)
|
|
||||||
|
|
||||||
|
|
||||||
## Copyleft
|
|
||||||
|
|
||||||
- NeoDB software code is licensed under AGPL, please check it if you plan to provide a service with modified code.
|
|
||||||
- If you publish a work (e.g. application, website or service) utilizing NeoDB software or API, you may use "NeoDB" and its visual identity in your work, but please refrain from any usage that may imply the affiliation with NeoDB software and team.
|
|
||||||
- If you publish such work to serve others, we do appreciate you clearly mention the usage of NeoDB in your work, provide descriptions and links to NeoDB software (neodb.net) and service providers (e.g. neodb.social), and guide your users to support them directly.
|
|
||||||
|
|
||||||
|
|
||||||
## Donation
|
|
||||||
|
|
||||||
If you appreciate this project, please help spread the words, and consider sponsoring the service providers. Your support is essential to keep these services free, open-sourced and stay committed to their mission. You may donate to the flagship instance, NeoDB.social, through the link below; or check the instance you use for their details about donations.
|
|
||||||
|
|
||||||
[](https://ko-fi.com/neodb)
|
|
||||||
|
|
||||||
|
|
||||||
## Feedback
|
|
||||||
|
|
||||||
Follow us on the Fediverse or join our Discord community to share your ideas, questions, or creations.
|
|
||||||
|
|
||||||
[](https://mastodon.online/@neodb)
|
|
||||||
[](https://mastodon.social/@neodb)
|
|
||||||
[](https://discord.gg/QBHkrV8bxK)
|
|
||||||
|
|
@ -1,90 +0,0 @@
|
||||||
Install
|
|
||||||
=======
|
|
||||||
|
|
||||||
For small and medium NeoDB instances, it's recommended to deploy as a container cluster with Docker Compose. To run a large instance, please see [scaling up](configuration.md#scaling-up) for some tips.
|
|
||||||
|
|
||||||
## Install docker compose
|
|
||||||
|
|
||||||
Follow [official instructions](https://docs.docker.com/compose/install/) to install Docker Compose if not yet.
|
|
||||||
|
|
||||||
Please verify its version is 2.x or higher before next step:
|
|
||||||
|
|
||||||
```
|
|
||||||
docker compose version
|
|
||||||
```
|
|
||||||
|
|
||||||
The rest of this doc assumes you can run docker commands without `sudo`, to verify that:
|
|
||||||
|
|
||||||
```
|
|
||||||
docker run --rm hello-world
|
|
||||||
```
|
|
||||||
|
|
||||||
Follow [official instructions](https://docs.docker.com/engine/install/linux-postinstall/) if it's not enabled, or use `sudo` to run commands in this doc.
|
|
||||||
|
|
||||||
|
|
||||||
## Prepare configuration files
|
|
||||||
- create a folder for configuration, eg ~/mysite/config
|
|
||||||
- grab `compose.yml` and `neodb.env.example` from [latest release](https://github.com/neodb-social/neodb/releases)
|
|
||||||
- rename `neodb.env.example` to `.env`
|
|
||||||
|
|
||||||
|
|
||||||
## Set up .env file and web root
|
|
||||||
Change essential options like `NEODB_SITE_DOMAIN` in `.env` before starting the cluster for the first time. Changing them later may have unintended consequences, please make sure they are correct before exposing the service externally.
|
|
||||||
|
|
||||||
- `NEODB_SITE_NAME` - name of your site
|
|
||||||
- `NEODB_SITE_DOMAIN` - domain name of your site
|
|
||||||
- `NEODB_SECRET_KEY` - encryption key of session data
|
|
||||||
- `NEODB_DATA` is the path to store db/media/cache, it's `../data` by default, but can be any path that's writable
|
|
||||||
- `NEODB_DEBUG` - set to `False` for production deployment
|
|
||||||
- `NEODB_PREFERRED_LANGUAGES` - preferred languages when importing titles from 3rd party sites like TMDB and Steam, comma-separated list of ISO-639-1 two-letter codes, 'en,zh' by default.
|
|
||||||
|
|
||||||
Optionally, `robots.txt` and `logo.png` may be placed under `$NEODB_DATA/www-root/`.
|
|
||||||
|
|
||||||
See [neodb.env.example](https://raw.githubusercontent.com/neodb-social/neodb/main/neodb.env.example) and [configuration](configuration.md) for more options
|
|
||||||
|
|
||||||
|
|
||||||
## Start container
|
|
||||||
|
|
||||||
in the folder with `compose.yml` and `.env`, execute as the user you just created:
|
|
||||||
```
|
|
||||||
docker compose --profile production pull
|
|
||||||
docker compose --profile production up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Starting up for the first time might take a few minutes, depending on download speed, use the following commands for status and logs:
|
|
||||||
```
|
|
||||||
docker compose ps
|
|
||||||
docker compose --profile production logs -f
|
|
||||||
```
|
|
||||||
|
|
||||||
In a few seconds, the site should be up at 127.0.0.1:8000 , you may check it with:
|
|
||||||
```
|
|
||||||
curl http://localhost:8000/nodeinfo/2.0/
|
|
||||||
```
|
|
||||||
|
|
||||||
JSON response will be returned if the server is up and running:
|
|
||||||
```
|
|
||||||
{"version": "2.0", "software": {"name": "neodb", "version": "0.8-dev"}, "protocols": ["activitypub", "neodb"], "services": {"outbound": [], "inbound": []}, "usage": {"users": {"total": 1}, "localPosts": 0}, "openRegistrations": true, "metadata": {}}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Make the site available publicly
|
|
||||||
|
|
||||||
Next step is to expose `http://127.0.0.1:8000` to external network as `https://yourdomain.tld` (NeoDB requires `https`). There are many ways to do it, you may use nginx or caddy as a reverse proxy server with an SSL cert configured, or configure a tunnel provider like cloudflared to do the same. Once done, you may check it with:
|
|
||||||
|
|
||||||
```
|
|
||||||
curl https://yourdomain.tld/nodeinfo/2.0/
|
|
||||||
```
|
|
||||||
|
|
||||||
You should see the same JSON response as above, and the site is now accessible to the public.
|
|
||||||
|
|
||||||
|
|
||||||
## Register an account and make it admin
|
|
||||||
|
|
||||||
Open `https://yourdomain.tld` in your browser and register an account, assuming username `admin`, run the following command to make it super user
|
|
||||||
|
|
||||||
```
|
|
||||||
docker compose --profile production run --rm shell neodb-manage user --super admin
|
|
||||||
```
|
|
||||||
|
|
||||||
Now your instance should be ready to serve. More tweaks are available, see [configuration](configuration.md) for options.
|
|
||||||
|
|
@ -1,114 +0,0 @@
|
||||||
Catalog
|
|
||||||
=======
|
|
||||||
|
|
||||||
Data Models
|
|
||||||
-----------
|
|
||||||
all types of catalog items inherits from `Item` which stores as multi-table django model.
|
|
||||||
one `Item` may have multiple `ExternalResource`s, each represents one page on an external site
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
classDiagram
|
|
||||||
class Item {
|
|
||||||
<<abstract>>
|
|
||||||
}
|
|
||||||
Item <|-- Album
|
|
||||||
class Album {
|
|
||||||
+String barcode
|
|
||||||
+String Douban_ID
|
|
||||||
+String Spotify_ID
|
|
||||||
}
|
|
||||||
Item <|-- Game
|
|
||||||
class Game {
|
|
||||||
+String Steam_ID
|
|
||||||
}
|
|
||||||
Item <|-- Podcast
|
|
||||||
class Podcast {
|
|
||||||
+String feed_url
|
|
||||||
+String Apple_ID
|
|
||||||
}
|
|
||||||
Item <|-- Performance
|
|
||||||
Item <|-- Work
|
|
||||||
class Work {
|
|
||||||
+String Douban_Work_ID
|
|
||||||
+String Goodreads_Work_ID
|
|
||||||
}
|
|
||||||
Item <|-- Edition
|
|
||||||
Item <|-- Series
|
|
||||||
|
|
||||||
Series *-- Work
|
|
||||||
Work *-- Edition
|
|
||||||
|
|
||||||
class Series {
|
|
||||||
+String Goodreads_Series_ID
|
|
||||||
}
|
|
||||||
class Work {
|
|
||||||
+String Douban_ID
|
|
||||||
+String Goodreads_ID
|
|
||||||
}
|
|
||||||
class Edition{
|
|
||||||
+String ISBN
|
|
||||||
+String Douban_ID
|
|
||||||
+String Goodreads_ID
|
|
||||||
+String GoogleBooks_ID
|
|
||||||
}
|
|
||||||
|
|
||||||
Item <|-- Movie
|
|
||||||
Item <|-- TVShow
|
|
||||||
Item <|-- TVSeason
|
|
||||||
Item <|-- TVEpisode
|
|
||||||
TVShow *-- TVSeason
|
|
||||||
TVSeason *-- TVEpisode
|
|
||||||
|
|
||||||
class TVShow{
|
|
||||||
+String IMDB_ID
|
|
||||||
+String TMDB_ID
|
|
||||||
}
|
|
||||||
class TVSeason{
|
|
||||||
+String Douban_ID
|
|
||||||
+String TMDB_ID
|
|
||||||
}
|
|
||||||
class TVEpisode{
|
|
||||||
+String IMDB_ID
|
|
||||||
+String TMDB_ID
|
|
||||||
}
|
|
||||||
class Movie{
|
|
||||||
+String Douban_ID
|
|
||||||
+String IMDB_ID
|
|
||||||
+String TMDB_ID
|
|
||||||
}
|
|
||||||
|
|
||||||
Item <|-- Collection
|
|
||||||
|
|
||||||
ExternalResource --* Item
|
|
||||||
class ExternalResource {
|
|
||||||
+enum site
|
|
||||||
+url: string
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Add a new site
|
|
||||||
--------------
|
|
||||||
|
|
||||||
- If official API is available for the site, it should be the preferred way to get data.
|
|
||||||
- add a new value to `IdType` and `SiteName` in `catalog/common/models.py`
|
|
||||||
- add a new file in `catalog/sites/`, a new class inherits `AbstractSite`, with:
|
|
||||||
* `SITE_NAME`
|
|
||||||
* `ID_TYPE`
|
|
||||||
* `URL_PATTERNS`
|
|
||||||
* `WIKI_PROPERTY_ID` (not used now)
|
|
||||||
* `DEFAULT_MODEL` (unless specified in `scrape()` return val)
|
|
||||||
* a classmethod `id_to_url()`
|
|
||||||
* a method `scrape()` returns a `ResourceContent` object
|
|
||||||
* `BasicDownloader` or `ProxiedDownloader` can used to download website content or API data. e.g. `content = BasicDownloader(url).download().html()`
|
|
||||||
* check out existing files in `catalog/sites/` for more examples
|
|
||||||
- add an import in `catalog/sites/__init__.py`
|
|
||||||
- add some tests to `catalog/<folder>/tests.py` according to site type
|
|
||||||
+ add `DOWNLOADER_SAVEDIR = '/tmp'` to `settings.py` can save all response to /tmp
|
|
||||||
+ run `neodb-manage cat <url>` for debugging or saving response file to `/tmp`. Detailed code of `cat` is in `catalog/management/commands/cat.py`
|
|
||||||
+ move captured response file to `test_data/`, except large/images files. Or if have to, replace it with a smallest version (e.g. 1x1 pixel / 1s audio)
|
|
||||||
+ add `@use_local_response` decorator to test methods that should pick up these responses (if `BasicDownloader` or `ProxiedDownloader` is used)
|
|
||||||
- run all the tests and make sure they pass
|
|
||||||
- Command: `neodb-manage python3 manage.py test [--keepdb]`.
|
|
||||||
- See [this issue](https://github.com/neodb-social/neodb/issues/5) if `lxml.etree.ParserError` occurs on macOS.
|
|
||||||
- add a site UI label style to `common/static/scss/_sitelabel.scss`
|
|
||||||
- update documentation in [sites.md](../sites.md)
|
|
||||||
|
|
@ -1,131 +0,0 @@
|
||||||
# Federation
|
|
||||||
|
|
||||||
## Supported federation protocols and standards
|
|
||||||
|
|
||||||
- [ActivityPub](https://www.w3.org/TR/activitypub/) (Server-to-Server)
|
|
||||||
- [WebFinger](https://webfinger.net/)
|
|
||||||
- [Http Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures)
|
|
||||||
- [NodeInfo](https://nodeinfo.diaspora.software/)
|
|
||||||
|
|
||||||
## Supported FEPs
|
|
||||||
|
|
||||||
- [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md)
|
|
||||||
|
|
||||||
## NodeInfo
|
|
||||||
|
|
||||||
NeoDB instances can be identified from user agent string (`NeoDB/x.x (+https://example.org)`) and `protocols` in its nodeinfo, e.g. https://neodb.social/nodeinfo/2.0/ :
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"version": "2.0",
|
|
||||||
"software": {
|
|
||||||
"name": "neodb",
|
|
||||||
"version": "0.10.4.13",
|
|
||||||
"repository": "https://github.com/neodb-social/neodb",
|
|
||||||
"homepage": "https://neodb.net/"
|
|
||||||
},
|
|
||||||
"protocols": ["activitypub", "neodb"],
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## ActivityPub
|
|
||||||
|
|
||||||
NeoDB's ActivityPub implementation is based on [Takahē](https://jointakahe.org), with some change to enable interchange of additional information between NeoDB instances.
|
|
||||||
|
|
||||||
### Activity
|
|
||||||
|
|
||||||
NeoDB add additional fields to `Note` activity:
|
|
||||||
|
|
||||||
- `relatedWith` is a list of NeoDB specific activities which are associated with this `Note`. For each activity, `id` and `href` are both unique links to that activity, `withRegardTo` links to the catalog item, `attributedTo` links to the user, `type` is one of:
|
|
||||||
- `Status`, its `status` can be one of: `complete`, `progress`, `wishlist` and `dropped`
|
|
||||||
- `Rating`, its `value` is rating grade (int, 1-10), `worst` is always 1, `best` is always 10
|
|
||||||
- `Comment`, its `content` is comment text
|
|
||||||
- `Review`, its `name` is review title, `content` is its body, `mediaType` is always `text/markdown` for now
|
|
||||||
- `Note`, its `content` is note text
|
|
||||||
- `tag` is used to store list of NeoDB catalog items, which are related with this activity. `type` of NeoDB catalog item can be one of `Edition`, `Movie`, `TVShow`, `TVSeason`, `TVEpisode`, `Album`, `Game`, `Podcast`, `PodcastEpisode`, `Performance`, `PerformanceProduction`; href will be the link to that item.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"@context": ["https://www.w3.org/ns/activitystreams", {
|
|
||||||
"blurhash": "toot:blurhash",
|
|
||||||
"Emoji": "toot:Emoji",
|
|
||||||
"focalPoint": {
|
|
||||||
"@container": "@list",
|
|
||||||
"@id": "toot:focalPoint"
|
|
||||||
},
|
|
||||||
"Hashtag": "as:Hashtag",
|
|
||||||
"manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
|
|
||||||
"sensitive": "as:sensitive",
|
|
||||||
"toot": "http://joinmastodon.org/ns#",
|
|
||||||
"votersCount": "toot:votersCount",
|
|
||||||
"featured": {
|
|
||||||
"@id": "toot:featured",
|
|
||||||
"@type": "@id"
|
|
||||||
}
|
|
||||||
}, "https://w3id.org/security/v1"],
|
|
||||||
"id": "https://neodb.social/@april_long_face@neodb.social/posts/380919151408919488/",
|
|
||||||
"type": "Note",
|
|
||||||
"relatedWith": [{
|
|
||||||
"id": "https://neodb.social/p/5oyF0qRx96mKKmVpFzHtMM",
|
|
||||||
"type": "Status",
|
|
||||||
"status": "complete",
|
|
||||||
"withRegardTo": "https://neodb.social/movie/7hfF7d0aFMaqHpFjUpq4zR",
|
|
||||||
"attributedTo": "https://neodb.social/@april_long_face@neodb.social/",
|
|
||||||
"href": "https://neodb.social/p/5oyF0qRx96mKKmVpFzHtMM",
|
|
||||||
"published": "2024-11-17T10:16:42.745240+00:00",
|
|
||||||
"updated": "2024-11-17T10:16:42.750917+00:00"
|
|
||||||
}, {
|
|
||||||
"id": "https://neodb.social/p/47cJnbQTkbSSN2izLwQMjo",
|
|
||||||
"type": "Comment",
|
|
||||||
"withRegardTo": "https://neodb.social/movie/7hfF7d0aFMaqHpFjUpq4zR",
|
|
||||||
"attributedTo": "https://neodb.social/@april_long_face@neodb.social/",
|
|
||||||
"content": "Broadway cin\u00e9math\u00e8que, at least I laughed hard.",
|
|
||||||
"href": "https://neodb.social/p/47cJnbQTkbSSN2izLwQMjo",
|
|
||||||
"published": "2024-11-17T10:16:42.745240+00:00",
|
|
||||||
"updated": "2024-11-17T10:16:42.777276+00:00"
|
|
||||||
}, {
|
|
||||||
"id": "https://neodb.social/p/3AyYu974qo6OU09AAsPweQ",
|
|
||||||
"type": "Rating",
|
|
||||||
"best": 10,
|
|
||||||
"value": 7,
|
|
||||||
"withRegardTo": "https://neodb.social/movie/7hfF7d0aFMaqHpFjUpq4zR",
|
|
||||||
"worst": 1,
|
|
||||||
"attributedTo": "https://neodb.social/@april_long_face@neodb.social/",
|
|
||||||
"href": "https://neodb.social/p/3AyYu974qo6OU09AAsPweQ",
|
|
||||||
"published": "2024-11-17T10:16:42.784220+00:00",
|
|
||||||
"updated": "2024-11-17T10:16:42.786458+00:00"
|
|
||||||
}],
|
|
||||||
"attributedTo": "https://neodb.social/@april_long_face@neodb.social/",
|
|
||||||
"content": "<p>\u770b\u8fc7 <a href=\"https://neodb.social/~neodb~/movie/7hfF7d0aFMaqHpFjUpq4zR\" rel=\"nofollow\">\u963f\u8bfa\u62c9</a> \ud83c\udf15\ud83c\udf15\ud83c\udf15\ud83c\udf17\ud83c\udf11 <br>Broadway cin\u00e9math\u00e8que, at least I laughed hard.</p><p><a href=\"https://neodb.social/tags/\u6211\u770b\u6211\u542c\u6211\u8bfb/\" class=\"mention hashtag\" rel=\"tag\">#\u6211\u770b\u6211\u542c\u6211\u8bfb</a></p>",
|
|
||||||
"published": "2024-11-17T10:16:42.745Z",
|
|
||||||
"sensitive": false,
|
|
||||||
"tag": [{
|
|
||||||
"type": "Hashtag",
|
|
||||||
"href": "https://neodb.social/tags/\u6211\u770b\u6211\u542c\u6211\u8bfb/",
|
|
||||||
"name": "#\u6211\u770b\u6211\u542c\u6211\u8bfb"
|
|
||||||
}, {
|
|
||||||
"type": "Movie",
|
|
||||||
"href": "https://neodb.social/movie/7hfF7d0aFMaqHpFjUpq4zR",
|
|
||||||
"image": "https://neodb.social/m/item/doubanmovie/2024/09/13/a30bf2f3-4f79-43ef-b22f-58ebc3fd8aae.jpg",
|
|
||||||
"name": "Anora"
|
|
||||||
}],
|
|
||||||
"to": ["https://www.w3.org/ns/activitystreams#Public"],
|
|
||||||
"updated": "2024-11-17T10:16:42.750Z",
|
|
||||||
"url": "https://neodb.social/@april_long_face/posts/380919151408919488/"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This is not ideal but a practical manner to pass along additional information between NeoDB instances and other ActivityPub servers. We have some ideas for improvements, but are open to more suggestions.
|
|
||||||
|
|
||||||
|
|
||||||
### Relay
|
|
||||||
|
|
||||||
NeoDB instances may share public rating and reviews with a default relay, which is currently `https://relay.neodb.net`. This relay is used to propagate public activities and catalog information between NeoDB instances.
|
|
||||||
|
|
||||||
Owner of each instance may choose to turn this off in their admin settings.
|
|
||||||
|
|
||||||
|
|
||||||
## ATProto
|
|
||||||
|
|
||||||
NeoDB is not a PDS itself currently, but can interact with PDS to import user's social graph, and send status updates. So technically NeoDB does not do full federation in ATProto, but NeoDB will handle some side effect from federation, e.g. when user logging in via ATProto handle, NeoDB will resolve user's DID and store it, and will attempt further operation with the DID, and update user's handle if that's changed, and use the corresponding PDS for that handle; user may still have to login NeoDB again with their Bluesky app password, since the change of PDS may invalidates previous app password.
|
|
||||||
|
|
@ -1,99 +0,0 @@
|
||||||
Journal
|
|
||||||
=======
|
|
||||||
|
|
||||||
|
|
||||||
Data Model
|
|
||||||
----------
|
|
||||||
```mermaid
|
|
||||||
classDiagram
|
|
||||||
User .. Piece
|
|
||||||
|
|
||||||
class Piece {
|
|
||||||
+User owner
|
|
||||||
+int visibility
|
|
||||||
}
|
|
||||||
|
|
||||||
class Item {
|
|
||||||
+str title
|
|
||||||
+str brief
|
|
||||||
-enum type
|
|
||||||
}
|
|
||||||
Piece <|-- Content
|
|
||||||
Item .. Content
|
|
||||||
class Content {
|
|
||||||
+Item target
|
|
||||||
}
|
|
||||||
Content <|-- Rating
|
|
||||||
class Rating {
|
|
||||||
+int grade
|
|
||||||
}
|
|
||||||
Content <|-- Review
|
|
||||||
class Review {
|
|
||||||
+str title
|
|
||||||
+str body
|
|
||||||
}
|
|
||||||
Content <|-- Comment
|
|
||||||
class Comment {
|
|
||||||
+str text
|
|
||||||
}
|
|
||||||
Content <|-- Note
|
|
||||||
class Note {
|
|
||||||
+str title
|
|
||||||
+str content
|
|
||||||
+enum progress_type
|
|
||||||
+str progress_value
|
|
||||||
|
|
||||||
}
|
|
||||||
Content <|-- Reply
|
|
||||||
class Reply {
|
|
||||||
+Content reply_to
|
|
||||||
}
|
|
||||||
Piece <|-- List
|
|
||||||
class List{
|
|
||||||
+ListItem[] items
|
|
||||||
}
|
|
||||||
Item .. ListItem
|
|
||||||
List *-- ListItem
|
|
||||||
class ListItem {
|
|
||||||
+int position
|
|
||||||
+Item item
|
|
||||||
+Dict metadata
|
|
||||||
}
|
|
||||||
List <|-- Collection
|
|
||||||
Item .. Collection
|
|
||||||
class Collection {
|
|
||||||
+str title
|
|
||||||
+str brief
|
|
||||||
+Bool collabrative
|
|
||||||
}
|
|
||||||
List <|-- Tag
|
|
||||||
class Tag {
|
|
||||||
+str title
|
|
||||||
}
|
|
||||||
List <|-- Shelf
|
|
||||||
class Shelf {
|
|
||||||
+Enum type
|
|
||||||
}
|
|
||||||
User .. ShelfLogManager
|
|
||||||
class ShelfLogManager {
|
|
||||||
+User owner
|
|
||||||
+ShelfLogEntry[] logs
|
|
||||||
}
|
|
||||||
ShelfLogManager *-- ShelfLogEntry
|
|
||||||
class ShelfLogEntry {
|
|
||||||
+Item item
|
|
||||||
+Shelf shelf
|
|
||||||
+DateTime timestamp
|
|
||||||
}
|
|
||||||
ShelfLogEntry .. Item
|
|
||||||
ShelfLogEntry .. Shelf
|
|
||||||
|
|
||||||
Shelf *-- ShelfItem
|
|
||||||
ListItem <|-- ShelfItem
|
|
||||||
|
|
||||||
ListItem <|-- TagItem
|
|
||||||
ListItem <|-- CollectionItem
|
|
||||||
|
|
||||||
Tag *-- TagItem
|
|
||||||
Collection *-- CollectionItem
|
|
||||||
```
|
|
||||||
|
|
@ -1,100 +0,0 @@
|
||||||
Social
|
|
||||||
======
|
|
||||||
|
|
||||||
Data Modal
|
|
||||||
----------
|
|
||||||
```mermaid
|
|
||||||
classDiagram
|
|
||||||
User .. Piece
|
|
||||||
|
|
||||||
class Piece {
|
|
||||||
+User owner
|
|
||||||
+int visibility
|
|
||||||
}
|
|
||||||
User .. Activity
|
|
||||||
class Activity {
|
|
||||||
+User owner
|
|
||||||
+int visibility
|
|
||||||
+Piece action_object
|
|
||||||
}
|
|
||||||
Activity .. Piece
|
|
||||||
Activity .. Item
|
|
||||||
class Item {
|
|
||||||
+str title
|
|
||||||
+str brief
|
|
||||||
-enum type
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Activities
|
|
||||||
----------
|
|
||||||
Activity data may be used for:
|
|
||||||
1. time line view of user and her friends
|
|
||||||
2. chronological view of user's action about an item
|
|
||||||
3. ActivityStreams `OrderedCollection` for ActivityPub
|
|
||||||
|
|
||||||
However, 2 is currently implemented separately via `ShelfLogManager` in `journal` app, because users may want to change these records manually.
|
|
||||||
|
|
||||||
Local Timeline
|
|
||||||
--------------
|
|
||||||
| Local Timeline Activities | action object class |
|
|
||||||
| ------------------------- | ------------------- |
|
|
||||||
| Add an Item to Shelf | ShelfMember |
|
|
||||||
| Create a Collection | Collection |
|
|
||||||
| Like a Collection | Like |
|
|
||||||
| Create a Review | Review |
|
|
||||||
|
|
||||||
|
|
||||||
Activity Streams
|
|
||||||
----------------
|
|
||||||
These are list of activities should be either shown in the site or delivered as ActivityStreams or both:
|
|
||||||
|
|
||||||
- `Add` / `Remove` an *Item* to / from a *List*:
|
|
||||||
+ add / remove *Item* to / from a user *Collection*
|
|
||||||
+ mark *Item* as wishlist / progress / complete, which are essentially add to / remove from user's predefined *Collection*
|
|
||||||
- `Create` / `Update` / `Delete` a user *Collection*
|
|
||||||
- `Create` / `Update` / `Delete` a *Content* with an `Object Link` to *Item*
|
|
||||||
+ `Create` / `Update` / `Delete` a *Comment* or *Review*
|
|
||||||
+ `Create` / `Update` / `Delete` a *Quote* or *Note*
|
|
||||||
- `Create` / `Update` / `Delete` a *Reply* to another *Content*
|
|
||||||
- `Announce` / `Like` a *Content* / *Collection*, or `Undo` that
|
|
||||||
- Social Graph interaction
|
|
||||||
+ `Follow`/`Unfo` `Follow`
|
|
||||||
+ `Accept`/`Reject`
|
|
||||||
+ `Block`/`Undo` `Block`
|
|
||||||
|
|
||||||
Supporting these activities above will be essential to a reasonable ActivityPub server implementation.
|
|
||||||
|
|
||||||
There are additional activities not made into ActivityPub MVP but technically possible to support in future:
|
|
||||||
- `Create` / `Update` / `Delete` a *Content* in different flavors, without link to *Item*
|
|
||||||
* `Note` or `Article` without link to *Item*
|
|
||||||
* DM (`Note`)
|
|
||||||
* `Question`
|
|
||||||
* `Article`
|
|
||||||
* `Page`
|
|
||||||
* `Image`
|
|
||||||
* `Audio`
|
|
||||||
* `Video`
|
|
||||||
* `Event`
|
|
||||||
- `Add` `Content` to / `Remove` `Content` from a user *Collection*
|
|
||||||
* *Pin* / *Unpin*
|
|
||||||
- `Move`/`Delete` account
|
|
||||||
|
|
||||||
ActivityPub
|
|
||||||
-----------
|
|
||||||
|
|
||||||
TBA
|
|
||||||
|
|
||||||
References:
|
|
||||||
- https://www.w3.org/TR/activitypub/
|
|
||||||
- https://www.w3.org/TR/activitystreams-core/
|
|
||||||
- https://www.w3.org/TR/activitystreams-vocabulary/
|
|
||||||
- https://www.w3.org/TR/json-ld/
|
|
||||||
- https://codeberg.org/fediverse/fep/src/branch/main/feps/fep-e232.md
|
|
||||||
- https://socialhub.activitypub.rocks/t/guide-for-new-activitypub-implementers/479
|
|
||||||
- https://docs.joinmastodon.org/spec/activitypub/
|
|
||||||
- https://docs.joinbookwyrm.com/activitypub.html
|
|
||||||
- https://github.com/Podcastindex-org/podcast-namespace/blob/main/proposal-docs/social/social.md#socialinteract-element
|
|
||||||
- https://dev.funkwhale.audio/funkwhale/funkwhale/-/tree/develop/docs/developer_documentation/federation
|
|
||||||
- https://github.com/inventaire/inventaire/issues/187
|
|
||||||
- https://github.com/inventaire/inventaire/issues/533
|
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
Our Story
|
|
||||||
=========
|
|
||||||
|
|
||||||
Established in September 2021, NeoDB is an [open source project](https://github.com/neodb-social/neodb) and [free service](https://neodb.social/) to help users around the world share and discover reviews and ratings of books, movies, music and games in Fediverse without concern of tracking or censorship.
|
|
||||||
|
|
||||||
|
|
||||||
The community that we serve today are mostly those previously used social networks in China to share their reviews and ratings for book, movie and music. By doing so for the last 15 years, they have made these reviews a rich form of archive that is the container of personal stories, intellectual debates, political opinions, and more. However, in recent years many of them got suppressed and banned due to increasing censorship and authoritarian laws in China. Many users have lost trust in proprietary software and services in China. NeoDB started as a humble effort to build open source software and services to help these internet refugees recreate their social network identity, regain control of their own content, and share their discoveries and opinions with their friends freely in Fediverse and, thanks to the openness of ActivityPub, the whole world.
|
|
||||||
|
|
||||||
|
|
||||||
As we’ve gained some [initial attraction](https://blog.joinmastodon.org/2021/12/mastodon-recap-2021/), we expect to continue supporting this fast growing community with better portability, security and user experience, we also hope to connect our community with broader Fediverse users with diverse backgrounds and serve both via enhanced a16y and i18n, decentralized architecture and interchangeable data format and protocols on catalogue information and user generated content. Your support and donation will help us operate continuously and develop towards this vision with more confidence.
|
|
||||||
|
|
||||||
|
|
||||||
NeoDB is a fork of [NiceDB](https://github.com/doubaniux/boofilsic), our code is actively being developed and is open-sourced under the AGPL-v3 license. We highly welcome contributions in code, design, and localization.
|
|
||||||
|
|
||||||
|
|
||||||
Please follow us on Fediverse([en](https://mastodon.online/@neodb)|[cn](https://mastodon.social/@neodb)) / [Bluesky](https://bsky.app/profile/neodb.net) / [Twitter](https://x.com/NeoDBsocial), and help spread the words. Thank you for the support!
|
|
||||||
|
|
@ -1,67 +0,0 @@
|
||||||
{
|
|
||||||
"version": "1.0",
|
|
||||||
"servers": [
|
|
||||||
{
|
|
||||||
"host": "neodb.social",
|
|
||||||
"description": "Flagship instance, managed by NeoDB developers.",
|
|
||||||
"label": [
|
|
||||||
"flagship"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"zh",
|
|
||||||
"en"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "NeoDB experimental",
|
|
||||||
"host": "eggplant.place",
|
|
||||||
"description": "Instance running development version of NeoDB software, which may have newer features and occationally bugs, managed by NeoDB developers.",
|
|
||||||
"label": [
|
|
||||||
"beta"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"en"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "ReviewDB",
|
|
||||||
"host": "neodb.social",
|
|
||||||
"admin": [
|
|
||||||
"@shlee@aus.social"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"en"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "Minreol",
|
|
||||||
"host": "minreol.dk",
|
|
||||||
"admin": [
|
|
||||||
"@pmakholm@norrebro.space"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"da"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "CasDB",
|
|
||||||
"host": "db.casually.cat",
|
|
||||||
"admin": [
|
|
||||||
"@casuallynoted@casually.cat"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"en"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "KevGa-NeoDB",
|
|
||||||
"host": "neodb.kevga.de",
|
|
||||||
"admin": [
|
|
||||||
"@lorker@mastodon.kevga.de"
|
|
||||||
],
|
|
||||||
"language": [
|
|
||||||
"de"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
@ -1,21 +0,0 @@
|
||||||
# Servers
|
|
||||||
|
|
||||||
|
|
||||||
## Community instances
|
|
||||||
|
|
||||||
NeoDB is not a single website. To use it, you need to sign up on an instance, that lets you connect with other people using NeoDB across Fediverse and Bluesky.
|
|
||||||
|
|
||||||
{servers}
|
|
||||||
|
|
||||||
JSON version of this list is also available [here](servers.json). If you are hosting a public instance of NeoDB and wish to share that with the community, please [edit this file](https://github.com/neodb-social/neodb/edit/main/docs/servers.json) and submit a pull request.
|
|
||||||
|
|
||||||
To host your own instance of NeoDB, see [installation guide](install.md).
|
|
||||||
|
|
||||||
|
|
||||||
## Public relay hosted by NeoDB developers
|
|
||||||
|
|
||||||
- `relay.neodb.net` - NeoDB instances may connect to this relay to send and receive public posts, this is to help share items, ratings and reviews in the network, more on this in [configuration doc](configuration.md).
|
|
||||||
|
|
||||||
|
|
||||||
## Honorable mention
|
|
||||||
- [NiceDB](https://nicedb.org) - the original instance, no longer open for registration.
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
# Supported Sites
|
|
||||||
|
|
||||||
the following sites are supported
|
|
||||||
|
|
||||||
|
|
||||||
| | import link for media type | import archive |
|
|
||||||
| ----------------------| ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
|
|
||||||
| Apple Music | Music (Album) | |
|
|
||||||
| Archive of Our Own | Book (Edition) | |
|
|
||||||
| Bandcamp | Music (Album) | |
|
|
||||||
| Bangumi | Book (Edition)<br>Music (Album)<br>Movie<br>TV (Season)<br>Game | |
|
|
||||||
| Board Game Geek | Game | |
|
|
||||||
| BooksTW 博客來 | Book (Edition) | |
|
|
||||||
| Discogs | Music (Album) | |
|
|
||||||
| Douban 豆瓣 | Book (Edition, Work)<br>Music (Album)<br>Movie<br>TV (Show, Season, Episode)<br>Game<br>Performance (Performance, Production) | Yes, upload [doufen](https://doufen.org) archive |
|
|
||||||
| Goodreads | Book (Edition, Work) | Yes, submit profile or shelf link |
|
|
||||||
| Google Books | Book (Edition) | |
|
|
||||||
| IGDB | Game | |
|
|
||||||
| IMDB | Movie<br>TV (Show, Episode) | |
|
|
||||||
| jjwxc 晋江文学城 | Book (Edition) | |
|
|
||||||
| Letterboxd | not supported (link in archive is mapped to TMDB) | Yes, upload exported archive |
|
|
||||||
| Qidian 起点 | Book (Edition) | |
|
|
||||||
| Spotify | Music (Album) | |
|
|
||||||
| Steam | Game | |
|
|
||||||
| The Movie Database | Movie<br>TV (Show, Season, Episode) | |
|
|
||||||
| ypshuo 阅评说 | Book (Edition) | |
|
|
||||||
| RSS link to a podcast | Podcast | Yes, upload OPML |
|
|
||||||
|
|
@ -1,14 +0,0 @@
|
||||||
:root {
|
|
||||||
--md-primary-fg-color: #0172ad;
|
|
||||||
--md-primary-fg-color--light: #0172ad;
|
|
||||||
--md-primary-fg-color--dark: #0172ad;
|
|
||||||
--md-accent-fg-color: #01aaff;
|
|
||||||
--md-accent-fg-color--light: #01aaff;
|
|
||||||
--md-accent-fg-color--dark: #01aaff;
|
|
||||||
font-family:system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, Helvetica, Arial, "Helvetica Neue", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
|
|
||||||
}
|
|
||||||
@media screen and (min-width: 76.25em) {
|
|
||||||
.md-nav__title {
|
|
||||||
display: none;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
5
docs/templates/main.html
vendored
5
docs/templates/main.html
vendored
|
|
@ -1,5 +0,0 @@
|
||||||
{% extends "base.html" %}
|
|
||||||
{% block extrahead %}
|
|
||||||
<link rel="me" href="https://mastodon.online/@neodb">
|
|
||||||
<link rel="me" href="https://mastodon.social/@neodb">
|
|
||||||
{% endblock %}
|
|
||||||
|
|
@ -1,56 +0,0 @@
|
||||||
# Troubleshooting
|
|
||||||
|
|
||||||
|
|
||||||
## Tips
|
|
||||||
|
|
||||||
- `docker compose ps` to see if any service is down, (btw it's normal that `migration` is in `Exit 0` state)
|
|
||||||
- `docker compose run shell` to run a shell into the cluster; or `docker compose run root` for root shell, and `apt` is available if extra package needed
|
|
||||||
- see `Debug in Docker` in [development doc](development.md) for debugging tips
|
|
||||||
|
|
||||||
|
|
||||||
## Containers
|
|
||||||
|
|
||||||
a typical neodb cluster looks like:
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TB
|
|
||||||
web[[Your reverse proxy server with SSL]] --- neodb-nginx[nginx listening on localhost:8000]
|
|
||||||
subgraph Containers managed by compose.yml
|
|
||||||
neodb-nginx --- neodb-web
|
|
||||||
neodb-nginx --- takahe-web
|
|
||||||
neodb-worker --- typesense[(typesense)]
|
|
||||||
neodb-worker --- neodb-db[(neodb-db)]
|
|
||||||
neodb-worker --- redis[(redis)]
|
|
||||||
neodb-web --- typesense
|
|
||||||
neodb-web --- neodb-db
|
|
||||||
neodb-web --- redis
|
|
||||||
neodb-web --- takahe-db[(takahe-db)]
|
|
||||||
migration([migration]) --- neodb-db
|
|
||||||
migration --- takahe-db
|
|
||||||
takahe-web --- takahe-db
|
|
||||||
takahe-web --- redis
|
|
||||||
takahe-stator --- takahe-db
|
|
||||||
takahe-stator --- redis
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Data Folders
|
|
||||||
|
|
||||||
a typical neodb folder after starting up should look like:
|
|
||||||
|
|
||||||
```
|
|
||||||
mysite
|
|
||||||
├── data # neodb data folder, location can be changed via NEODB_DATA in .env
|
|
||||||
│ ├── neodb-db # neodb database
|
|
||||||
│ ├── neodb-media # uid must be 1000 (app user in docker image), chmod if not so
|
|
||||||
│ ├── redis # neodb/takahe cache
|
|
||||||
│ ├── takahe-cache # uid must be 33 (www-data user in docker image), chmod if not so
|
|
||||||
│ ├── takahe-db # neodb database
|
|
||||||
│ ├── takahe-media # uid must be 1000 (app user in docker image), chmod if not so
|
|
||||||
│ ├── typesense # neodb search index
|
|
||||||
│ └── www-root # neodb web root for robots.txt, logo.png and etc
|
|
||||||
└── config
|
|
||||||
├── compose.yml # copied from neodb release
|
|
||||||
└── .env # your configuration, see neodb.env.example
|
|
||||||
```
|
|
||||||
|
|
@ -1,21 +0,0 @@
|
||||||
Upgrade
|
|
||||||
=======
|
|
||||||
|
|
||||||
Check the [release notes](https://github.com/neodb-social/neodb/releases), update `compose.yml` and `.env` as instructed.
|
|
||||||
|
|
||||||
If there is `compose.override.yml`, make sure it's compatible with the updated `compose.yml`.
|
|
||||||
|
|
||||||
Pull the latest container image
|
|
||||||
```bash
|
|
||||||
docker compose --profile production pull
|
|
||||||
```
|
|
||||||
|
|
||||||
Restart the entire cluster:
|
|
||||||
```bash
|
|
||||||
docker compose --profile production up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Optionally, clean up old images:
|
|
||||||
```bash
|
|
||||||
docker system prune -af --volumes
|
|
||||||
```
|
|
||||||
|
|
@ -43,16 +43,17 @@ class LetterboxdImporter(Task):
|
||||||
except Exception:
|
except Exception:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def get_item_by_url(self, url):
|
@classmethod
|
||||||
|
def get_item_by_url(cls, url):
|
||||||
try:
|
try:
|
||||||
h = BasicDownloader(url).download().html()
|
h = BasicDownloader(url).download().html()
|
||||||
tu = h.xpath("//a[@data-track-action='TMDb']/@href")
|
tu = h.xpath("//a[@data-track-action='TMDB']/@href")
|
||||||
iu = h.xpath("//a[@data-track-action='IMDb']/@href")
|
iu = h.xpath("//a[@data-track-action='IMDb']/@href")
|
||||||
if not tu:
|
if not tu:
|
||||||
i = h.xpath('//span[@class="film-title-wrapper"]/a/@href')
|
i = h.xpath('//span[@class="film-title-wrapper"]/a/@href')
|
||||||
u2 = "https://letterboxd.com" + i[0] # type:ignore
|
u2 = "https://letterboxd.com" + i[0] # type:ignore
|
||||||
h = BasicDownloader(u2).download().html()
|
h = BasicDownloader(u2).download().html()
|
||||||
tu = h.xpath("//a[@data-track-action='TMDb']/@href")
|
tu = h.xpath("//a[@data-track-action='TMDB']/@href")
|
||||||
iu = h.xpath("//a[@data-track-action='IMDb']/@href")
|
iu = h.xpath("//a[@data-track-action='IMDb']/@href")
|
||||||
if not tu:
|
if not tu:
|
||||||
logger.error(f"Unknown TMDB for {url}")
|
logger.error(f"Unknown TMDB for {url}")
|
||||||
|
|
|
||||||
|
|
@ -178,7 +178,11 @@ class JournalSearchResult(SearchResult):
|
||||||
[],
|
[],
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
items = Item.objects.filter(pk__in=ids, is_deleted=False)
|
select = {f"id_{i}": f"id={i}" for i in ids}
|
||||||
|
order = [f"-id_{i}" for i in ids]
|
||||||
|
items = Item.objects.filter(pk__in=ids, is_deleted=False).extra(
|
||||||
|
select=select, order_by=order
|
||||||
|
)
|
||||||
items = [j for j in [i.final_item for i in items] if not j.is_deleted]
|
items = [j for j in [i.final_item for i in items] if not j.is_deleted]
|
||||||
return items
|
return items
|
||||||
|
|
||||||
|
|
@ -197,7 +201,9 @@ class JournalSearchResult(SearchResult):
|
||||||
],
|
],
|
||||||
[],
|
[],
|
||||||
)
|
)
|
||||||
ps = Piece.objects.filter(pk__in=ids)
|
select = {f"id_{i}": f"id={i}" for i in ids}
|
||||||
|
order = [f"-id_{i}" for i in ids]
|
||||||
|
ps = Piece.objects.filter(pk__in=ids).extra(select=select, order_by=order)
|
||||||
return ps
|
return ps
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
|
|
|
||||||
|
|
@ -163,7 +163,7 @@ class Mark:
|
||||||
log entries
|
log entries
|
||||||
log entry will be created when item is added to shelf
|
log entry will be created when item is added to shelf
|
||||||
log entry will be created when item is moved to another shelf
|
log entry will be created when item is moved to another shelf
|
||||||
log entry will be created when item is removed from shelf (TODO change this to DEFERRED shelf)
|
log entry will be created when item is removed from shelf
|
||||||
timestamp of log entry will be updated whenever created_time of shelfmember is updated
|
timestamp of log entry will be updated whenever created_time of shelfmember is updated
|
||||||
any log entry can be deleted by user arbitrarily
|
any log entry can be deleted by user arbitrarily
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,14 +5,14 @@ from django.core.validators import MaxValueValidator, MinValueValidator
|
||||||
from django.db import models
|
from django.db import models
|
||||||
from django.db.models import Avg, Count
|
from django.db.models import Avg, Count
|
||||||
|
|
||||||
from catalog.models import Item
|
from catalog.models import Item, Performance, TVShow
|
||||||
from takahe.utils import Takahe
|
from takahe.utils import Takahe
|
||||||
from users.models import APIdentity
|
from users.models import APIdentity
|
||||||
|
|
||||||
from .common import Content
|
from .common import Content
|
||||||
|
|
||||||
MIN_RATING_COUNT = 5
|
MIN_RATING_COUNT = 5
|
||||||
RATING_INCLUDES_CHILD_ITEMS = ["tvshow", "performance"]
|
RATING_INCLUDES_CHILD_ITEMS = [TVShow, Performance]
|
||||||
|
|
||||||
|
|
||||||
class Rating(Content):
|
class Rating(Content):
|
||||||
|
|
@ -73,10 +73,41 @@ class Rating(Content):
|
||||||
p.link_post_id(post.id)
|
p.link_post_id(post.id)
|
||||||
return p
|
return p
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def get_info_for_item(cls, item: Item) -> dict:
|
||||||
|
stat = Rating.objects.filter(grade__isnull=False)
|
||||||
|
if item.__class__ in RATING_INCLUDES_CHILD_ITEMS:
|
||||||
|
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
||||||
|
else:
|
||||||
|
stat = stat.filter(item=item)
|
||||||
|
stat = stat.values("grade").annotate(count=Count("grade"))
|
||||||
|
grades = [0] * 11
|
||||||
|
votes = 0
|
||||||
|
total = 0
|
||||||
|
for s in stat:
|
||||||
|
if s["grade"] and s["grade"] > 0 and s["grade"] < 11:
|
||||||
|
grades[s["grade"]] = s["count"]
|
||||||
|
total += s["count"] * s["grade"]
|
||||||
|
votes += s["count"]
|
||||||
|
if votes < MIN_RATING_COUNT:
|
||||||
|
return {"average": None, "count": votes, "distribution": [0] * 5}
|
||||||
|
else:
|
||||||
|
return {
|
||||||
|
"average": round(total / votes, 1),
|
||||||
|
"count": votes,
|
||||||
|
"distribution": [
|
||||||
|
100 * (grades[1] + grades[2]) // votes,
|
||||||
|
100 * (grades[3] + grades[4]) // votes,
|
||||||
|
100 * (grades[5] + grades[6]) // votes,
|
||||||
|
100 * (grades[7] + grades[8]) // votes,
|
||||||
|
100 * (grades[9] + grades[10]) // votes,
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_rating_for_item(item: Item) -> float | None:
|
def get_rating_for_item(item: Item) -> float | None:
|
||||||
stat = Rating.objects.filter(grade__isnull=False)
|
stat = Rating.objects.filter(grade__isnull=False)
|
||||||
if item.class_name in RATING_INCLUDES_CHILD_ITEMS:
|
if item.__class__ in RATING_INCLUDES_CHILD_ITEMS:
|
||||||
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
||||||
else:
|
else:
|
||||||
stat = stat.filter(item=item)
|
stat = stat.filter(item=item)
|
||||||
|
|
@ -86,7 +117,7 @@ class Rating(Content):
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_rating_count_for_item(item: Item) -> int:
|
def get_rating_count_for_item(item: Item) -> int:
|
||||||
stat = Rating.objects.filter(grade__isnull=False)
|
stat = Rating.objects.filter(grade__isnull=False)
|
||||||
if item.class_name in RATING_INCLUDES_CHILD_ITEMS:
|
if item.__class__ in RATING_INCLUDES_CHILD_ITEMS:
|
||||||
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
||||||
else:
|
else:
|
||||||
stat = stat.filter(item=item)
|
stat = stat.filter(item=item)
|
||||||
|
|
@ -96,7 +127,7 @@ class Rating(Content):
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_rating_distribution_for_item(item: Item):
|
def get_rating_distribution_for_item(item: Item):
|
||||||
stat = Rating.objects.filter(grade__isnull=False)
|
stat = Rating.objects.filter(grade__isnull=False)
|
||||||
if item.class_name in RATING_INCLUDES_CHILD_ITEMS:
|
if item.__class__ in RATING_INCLUDES_CHILD_ITEMS:
|
||||||
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
stat = stat.filter(item_id__in=item.child_item_ids + [item.pk])
|
||||||
else:
|
else:
|
||||||
stat = stat.filter(item=item)
|
stat = stat.filter(item=item)
|
||||||
|
|
|
||||||
|
|
@ -50,6 +50,10 @@ def html_to_text(h: str) -> str:
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def has_spoiler(s: str) -> bool:
|
||||||
|
return ">!" in s
|
||||||
|
|
||||||
|
|
||||||
def _spolier(s: str) -> str:
|
def _spolier(s: str) -> str:
|
||||||
sl = s.split(">!", 1)
|
sl = s.split(">!", 1)
|
||||||
if len(sl) == 1:
|
if len(sl) == 1:
|
||||||
|
|
|
||||||
|
|
@ -7,6 +7,7 @@ from django.db import connection, models
|
||||||
from django.utils import timezone
|
from django.utils import timezone
|
||||||
from django.utils.translation import gettext_lazy as _
|
from django.utils.translation import gettext_lazy as _
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
from polymorphic.models import PolymorphicManager
|
||||||
|
|
||||||
from catalog.models import Item, ItemCategory
|
from catalog.models import Item, ItemCategory
|
||||||
from takahe.utils import Takahe
|
from takahe.utils import Takahe
|
||||||
|
|
@ -310,6 +311,28 @@ _SHELF_LABELS = [
|
||||||
# grammatically problematic, for translation only
|
# grammatically problematic, for translation only
|
||||||
|
|
||||||
|
|
||||||
|
class ShelfMemberManager(PolymorphicManager):
|
||||||
|
def get_queryset(self):
|
||||||
|
from .comment import Comment
|
||||||
|
from .rating import Rating
|
||||||
|
|
||||||
|
rating_subquery = Rating.objects.filter(
|
||||||
|
owner_id=models.OuterRef("owner_id"), item_id=models.OuterRef("item_id")
|
||||||
|
).values("grade")[:1]
|
||||||
|
comment_subquery = Comment.objects.filter(
|
||||||
|
owner_id=models.OuterRef("owner_id"), item_id=models.OuterRef("item_id")
|
||||||
|
).values("text")[:1]
|
||||||
|
return (
|
||||||
|
super()
|
||||||
|
.get_queryset()
|
||||||
|
.annotate(
|
||||||
|
_rating_grade=models.Subquery(rating_subquery),
|
||||||
|
_comment_text=models.Subquery(comment_subquery),
|
||||||
|
_shelf_type=models.F("parent__shelf_type"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class ShelfMember(ListMember):
|
class ShelfMember(ListMember):
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
parent: models.ForeignKey["ShelfMember", "Shelf"]
|
parent: models.ForeignKey["ShelfMember", "Shelf"]
|
||||||
|
|
@ -318,6 +341,8 @@ class ShelfMember(ListMember):
|
||||||
"Shelf", related_name="members", on_delete=models.CASCADE
|
"Shelf", related_name="members", on_delete=models.CASCADE
|
||||||
)
|
)
|
||||||
|
|
||||||
|
objects = ShelfMemberManager()
|
||||||
|
|
||||||
class Meta:
|
class Meta:
|
||||||
unique_together = [["owner", "item"]]
|
unique_together = [["owner", "item"]]
|
||||||
indexes = [
|
indexes = [
|
||||||
|
|
@ -448,6 +473,15 @@ class ShelfMember(ListMember):
|
||||||
"content": content,
|
"content": content,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def save(self, *args, **kwargs):
|
||||||
|
try:
|
||||||
|
del self._shelf_type # type:ignore
|
||||||
|
del self._rating_grade # type:ignore
|
||||||
|
del self._comment_text # type:ignore
|
||||||
|
except AttributeError:
|
||||||
|
pass
|
||||||
|
return super().save(*args, **kwargs)
|
||||||
|
|
||||||
@cached_property
|
@cached_property
|
||||||
def sibling_comment(self) -> "Comment | None":
|
def sibling_comment(self) -> "Comment | None":
|
||||||
from .comment import Comment
|
from .comment import Comment
|
||||||
|
|
@ -470,19 +504,28 @@ class ShelfMember(ListMember):
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def shelf_label(self) -> str | None:
|
def shelf_label(self) -> str | None:
|
||||||
return ShelfManager.get_label(self.parent.shelf_type, self.item.category)
|
return ShelfManager.get_label(self.shelf_type, self.item.category)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def shelf_type(self):
|
def shelf_type(self):
|
||||||
return self.parent.shelf_type
|
try:
|
||||||
|
return getattr(self, "_shelf_type")
|
||||||
|
except AttributeError:
|
||||||
|
return self.parent.shelf_type
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def rating_grade(self):
|
def rating_grade(self):
|
||||||
return self.mark.rating_grade
|
try:
|
||||||
|
return getattr(self, "_rating_grade")
|
||||||
|
except AttributeError:
|
||||||
|
return self.mark.rating_grade
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def comment_text(self):
|
def comment_text(self):
|
||||||
return self.mark.comment_text
|
try:
|
||||||
|
return getattr(self, "_comment_text")
|
||||||
|
except AttributeError:
|
||||||
|
return self.mark.comment_text
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def tags(self):
|
def tags(self):
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
{% load i18n %}
|
{% load i18n %}
|
||||||
<aside class="grid__aside bottom">
|
<aside class="grid__aside bottom">
|
||||||
<p>
|
<details open>
|
||||||
{% trans 'Addtional filters may be added to query' %}
|
<summary>{% trans 'Search filters' %}</summary>
|
||||||
<li>
|
<li>
|
||||||
<code>status:complete</code> - {% trans 'filter by one of these statuses' %}: <code>wishlist</code> / <code>progress</code> / <code>complete</code> / <code>dropped</code>
|
<code>status:complete</code> - {% trans 'filter by one of these statuses' %}: <code>wishlist</code> / <code>progress</code> / <code>complete</code> / <code>dropped</code>
|
||||||
</li>
|
</li>
|
||||||
|
|
@ -29,5 +29,8 @@
|
||||||
<li>
|
<li>
|
||||||
<code>date:2022..2023</code> - {% trans 'filter by date range' %}
|
<code>date:2022..2023</code> - {% trans 'filter by date range' %}
|
||||||
</li>
|
</li>
|
||||||
</p>
|
<li>
|
||||||
|
<code>sort:date</code> - {% trans "sort by one of these fields" %}: <code>date</code> ({% trans "from new to old" %}) / <code>rating</code> ({% trans "from high to low" %})
|
||||||
|
</li>
|
||||||
|
</details>
|
||||||
</aside>
|
</aside>
|
||||||
|
|
|
||||||
|
|
@ -1,4 +1,5 @@
|
||||||
from .csv import *
|
from .csv import *
|
||||||
from .ndjson import *
|
from .ndjson import *
|
||||||
from .piece import *
|
from .piece import *
|
||||||
|
from .rating import *
|
||||||
from .search import *
|
from .search import *
|
||||||
|
|
|
||||||
|
|
@ -65,23 +65,34 @@ class ShelfTest(TestCase):
|
||||||
self.assertEqual(q1.members.all().count(), 0)
|
self.assertEqual(q1.members.all().count(), 0)
|
||||||
self.assertEqual(q2.members.all().count(), 0)
|
self.assertEqual(q2.members.all().count(), 0)
|
||||||
Mark(user.identity, book1).update(ShelfType.WISHLIST)
|
Mark(user.identity, book1).update(ShelfType.WISHLIST)
|
||||||
time.sleep(0.001) # add a little delay to make sure the timestamp is different
|
|
||||||
Mark(user.identity, book2).update(ShelfType.WISHLIST)
|
Mark(user.identity, book2).update(ShelfType.WISHLIST)
|
||||||
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book1)]
|
||||||
|
self.assertEqual(log, ["wishlist"])
|
||||||
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book2)]
|
||||||
|
self.assertEqual(log, ["wishlist"])
|
||||||
|
time.sleep(0.001) # add a little delay to make sure the timestamp is different
|
||||||
|
|
||||||
|
Mark(user.identity, book1).update(ShelfType.WISHLIST)
|
||||||
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book1)]
|
||||||
|
self.assertEqual(log, ["wishlist"])
|
||||||
time.sleep(0.001)
|
time.sleep(0.001)
|
||||||
|
|
||||||
self.assertEqual(q1.members.all().count(), 2)
|
self.assertEqual(q1.members.all().count(), 2)
|
||||||
Mark(user.identity, book1).update(ShelfType.PROGRESS)
|
Mark(user.identity, book1).update(ShelfType.PROGRESS)
|
||||||
time.sleep(0.001)
|
|
||||||
self.assertEqual(q1.members.all().count(), 1)
|
self.assertEqual(q1.members.all().count(), 1)
|
||||||
self.assertEqual(q2.members.all().count(), 1)
|
self.assertEqual(q2.members.all().count(), 1)
|
||||||
|
time.sleep(0.001)
|
||||||
|
|
||||||
self.assertEqual(len(Mark(user.identity, book1).all_post_ids), 2)
|
self.assertEqual(len(Mark(user.identity, book1).all_post_ids), 2)
|
||||||
log = shelf_manager.get_log_for_item(book1)
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book1)]
|
||||||
self.assertEqual(log.count(), 2)
|
|
||||||
|
self.assertEqual(log, ["wishlist", "progress"])
|
||||||
Mark(user.identity, book1).update(ShelfType.PROGRESS, metadata={"progress": 1})
|
Mark(user.identity, book1).update(ShelfType.PROGRESS, metadata={"progress": 1})
|
||||||
time.sleep(0.001)
|
time.sleep(0.001)
|
||||||
self.assertEqual(q1.members.all().count(), 1)
|
self.assertEqual(q1.members.all().count(), 1)
|
||||||
self.assertEqual(q2.members.all().count(), 1)
|
self.assertEqual(q2.members.all().count(), 1)
|
||||||
log = shelf_manager.get_log_for_item(book1)
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book1)]
|
||||||
self.assertEqual(log.count(), 2)
|
self.assertEqual(log, ["wishlist", "progress"])
|
||||||
self.assertEqual(len(Mark(user.identity, book1).all_post_ids), 2)
|
self.assertEqual(len(Mark(user.identity, book1).all_post_ids), 2)
|
||||||
|
|
||||||
# theses tests are not relevant anymore, bc we don't use log to track metadata changes
|
# theses tests are not relevant anymore, bc we don't use log to track metadata changes
|
||||||
|
|
@ -127,7 +138,8 @@ class ShelfTest(TestCase):
|
||||||
|
|
||||||
# test delete mark -> one more log
|
# test delete mark -> one more log
|
||||||
Mark(user.identity, book1).delete()
|
Mark(user.identity, book1).delete()
|
||||||
self.assertEqual(log.count(), 4)
|
log = [ll.shelf_type for ll in shelf_manager.get_log_for_item(book1)]
|
||||||
|
self.assertEqual(log, ["wishlist", "progress", "complete", None])
|
||||||
deleted_mark = Mark(user.identity, book1)
|
deleted_mark = Mark(user.identity, book1)
|
||||||
self.assertEqual(deleted_mark.shelf_type, None)
|
self.assertEqual(deleted_mark.shelf_type, None)
|
||||||
self.assertEqual(deleted_mark.tags, [])
|
self.assertEqual(deleted_mark.tags, [])
|
||||||
|
|
|
||||||
201
journal/tests/rating.py
Normal file
201
journal/tests/rating.py
Normal file
|
|
@ -0,0 +1,201 @@
|
||||||
|
from django.test import TestCase
|
||||||
|
|
||||||
|
from catalog.common.models import Item
|
||||||
|
from catalog.models import Edition, IdType, Movie, TVEpisode, TVSeason, TVShow
|
||||||
|
from journal.models.rating import Rating
|
||||||
|
from users.models import User
|
||||||
|
|
||||||
|
|
||||||
|
class RatingTest(TestCase):
|
||||||
|
databases = "__all__"
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
# Create 10 users
|
||||||
|
self.users = []
|
||||||
|
for i in range(10):
|
||||||
|
user = User.register(email=f"user{i}@example.com", username=f"user{i}")
|
||||||
|
self.users.append(user)
|
||||||
|
|
||||||
|
# Create a book
|
||||||
|
self.book = Edition.objects.create(
|
||||||
|
localized_title=[{"lang": "en", "text": "Test Book"}],
|
||||||
|
primary_lookup_id_type=IdType.ISBN,
|
||||||
|
primary_lookup_id_value="9780553283686",
|
||||||
|
author=["Test Author"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create a movie
|
||||||
|
self.movie = Movie.objects.create(
|
||||||
|
localized_title=[{"lang": "en", "text": "Test Movie"}],
|
||||||
|
primary_lookup_id_type=IdType.IMDB,
|
||||||
|
primary_lookup_id_value="tt1234567",
|
||||||
|
director=["Test Director"],
|
||||||
|
year=2020,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create a TV show with a season and episode
|
||||||
|
self.tvshow = TVShow.objects.create(
|
||||||
|
localized_title=[{"lang": "en", "text": "Test Show"}],
|
||||||
|
primary_lookup_id_type=IdType.IMDB,
|
||||||
|
primary_lookup_id_value="tt9876543",
|
||||||
|
)
|
||||||
|
self.tvseason = TVSeason.objects.create(
|
||||||
|
localized_title=[{"lang": "en", "text": "Season 1"}],
|
||||||
|
show=self.tvshow,
|
||||||
|
season_number=1,
|
||||||
|
)
|
||||||
|
self.tvepisode = TVEpisode.objects.create(
|
||||||
|
localized_title=[{"lang": "en", "text": "Episode 1"}],
|
||||||
|
season=self.tvseason,
|
||||||
|
episode_number=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_rating_basic(self):
|
||||||
|
"""Test basic rating functionality for a single item."""
|
||||||
|
# Add ratings for the book from all users
|
||||||
|
ratings = [7, 8, 9, 10, 8, 7, 6, 9, 10, 8]
|
||||||
|
|
||||||
|
for i, user in enumerate(self.users):
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.book, user.identity, ratings[i], visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Get rating info for the book
|
||||||
|
rating_info = Rating.get_info_for_item(self.book)
|
||||||
|
|
||||||
|
# Check rating count
|
||||||
|
self.assertEqual(rating_info["count"], 10)
|
||||||
|
|
||||||
|
# Check average rating - should be 8.2
|
||||||
|
expected_avg = sum(ratings) / len(ratings)
|
||||||
|
self.assertEqual(rating_info["average"], round(expected_avg, 1))
|
||||||
|
|
||||||
|
# Check distribution
|
||||||
|
# [1-2, 3-4, 5-6, 7-8, 9-10] buckets represented as percentages
|
||||||
|
expected_distribution = [0, 0, 10, 50, 40] # Based on our ratings
|
||||||
|
self.assertEqual(rating_info["distribution"], expected_distribution)
|
||||||
|
|
||||||
|
# Test individual user rating
|
||||||
|
user_rating = Rating.get_item_rating(self.book, self.users[0].identity)
|
||||||
|
self.assertEqual(user_rating, 7)
|
||||||
|
|
||||||
|
book = Item.objects.get(pk=self.book.pk)
|
||||||
|
self.assertEqual(book.rating, round(expected_avg, 1))
|
||||||
|
self.assertEqual(book.rating_count, 10)
|
||||||
|
self.assertEqual(book.rating_distribution, expected_distribution)
|
||||||
|
|
||||||
|
def test_rating_multiple_items(self):
|
||||||
|
"""Test ratings across multiple items."""
|
||||||
|
# Rate the movie with varying scores
|
||||||
|
movie_ratings = [3, 4, 5, 6, 7, 8, 9, 10, 2, 1]
|
||||||
|
|
||||||
|
for i, user in enumerate(self.users):
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.movie, user.identity, movie_ratings[i], visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Rate the TV show
|
||||||
|
tvshow_ratings = [10, 9, 8, 9, 10, 9, 8, 10, 9, 8]
|
||||||
|
|
||||||
|
for i, user in enumerate(self.users):
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.tvshow, user.identity, tvshow_ratings[i], visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Get rating info for both items
|
||||||
|
movie_info = Rating.get_info_for_item(self.movie)
|
||||||
|
tvshow_info = Rating.get_info_for_item(self.tvshow)
|
||||||
|
|
||||||
|
# Check counts
|
||||||
|
self.assertEqual(movie_info["count"], 10)
|
||||||
|
self.assertEqual(tvshow_info["count"], 10)
|
||||||
|
|
||||||
|
# Check averages
|
||||||
|
expected_movie_avg = sum(movie_ratings) / len(movie_ratings)
|
||||||
|
expected_tvshow_avg = sum(tvshow_ratings) / len(tvshow_ratings)
|
||||||
|
|
||||||
|
self.assertEqual(movie_info["average"], round(expected_movie_avg, 1))
|
||||||
|
self.assertEqual(tvshow_info["average"], round(expected_tvshow_avg, 1))
|
||||||
|
|
||||||
|
# Check distribution for movie
|
||||||
|
# [1-2, 3-4, 5-6, 7-8, 9-10] buckets
|
||||||
|
expected_movie_distribution = [
|
||||||
|
20,
|
||||||
|
20,
|
||||||
|
20,
|
||||||
|
20,
|
||||||
|
20,
|
||||||
|
] # Evenly distributed across buckets
|
||||||
|
self.assertEqual(movie_info["distribution"], expected_movie_distribution)
|
||||||
|
|
||||||
|
# Check distribution for TV show
|
||||||
|
# [1-2, 3-4, 5-6, 7-8, 9-10] buckets
|
||||||
|
expected_tvshow_distribution = [0, 0, 0, 30, 70] # High ratings only
|
||||||
|
self.assertEqual(tvshow_info["distribution"], expected_tvshow_distribution)
|
||||||
|
|
||||||
|
def test_rating_update_and_delete(self):
|
||||||
|
"""Test updating and deleting ratings."""
|
||||||
|
# Add initial ratings
|
||||||
|
for user in self.users[:5]:
|
||||||
|
Rating.update_item_rating(self.tvepisode, user.identity, 8, visibility=1)
|
||||||
|
|
||||||
|
# Check initial count
|
||||||
|
self.assertEqual(Rating.get_rating_count_for_item(self.tvepisode), 5)
|
||||||
|
|
||||||
|
# Update a rating
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.tvepisode, self.users[0].identity, 10, visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check that rating was updated
|
||||||
|
updated_rating = Rating.get_item_rating(self.tvepisode, self.users[0].identity)
|
||||||
|
self.assertEqual(updated_rating, 10)
|
||||||
|
|
||||||
|
# Delete a rating by setting it to None
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.tvepisode, self.users[1].identity, None, visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check that rating count decreased
|
||||||
|
self.assertEqual(Rating.get_rating_count_for_item(self.tvepisode), 4)
|
||||||
|
|
||||||
|
# Check that the rating was deleted
|
||||||
|
deleted_rating = Rating.get_item_rating(self.tvepisode, self.users[1].identity)
|
||||||
|
self.assertIsNone(deleted_rating)
|
||||||
|
|
||||||
|
def test_rating_minimum_count(self):
|
||||||
|
"""Test the minimum rating count threshold."""
|
||||||
|
# Add only 4 ratings to the book (below MIN_RATING_COUNT of 5)
|
||||||
|
for user in self.users[:4]:
|
||||||
|
Rating.update_item_rating(self.book, user.identity, 10, visibility=1)
|
||||||
|
|
||||||
|
# Check that get_rating_for_item returns None due to insufficient ratings
|
||||||
|
rating = Rating.get_rating_for_item(self.book)
|
||||||
|
self.assertIsNone(rating)
|
||||||
|
|
||||||
|
# Add one more rating to reach the threshold
|
||||||
|
Rating.update_item_rating(self.book, self.users[4].identity, 10, visibility=1)
|
||||||
|
|
||||||
|
# Now we should get a valid rating
|
||||||
|
rating = Rating.get_rating_for_item(self.book)
|
||||||
|
self.assertEqual(rating, 10.0)
|
||||||
|
|
||||||
|
def test_tvshow_rating_includes_children(self):
|
||||||
|
"""Test that TV show ratings include ratings from child items."""
|
||||||
|
# Rate the TV show directly
|
||||||
|
Rating.update_item_rating(self.tvshow, self.users[0].identity, 6, visibility=1)
|
||||||
|
|
||||||
|
# Rate the episode (which is a child of the TV show)
|
||||||
|
for i in range(1, 6): # Users 1-5
|
||||||
|
Rating.update_item_rating(
|
||||||
|
self.tvseason, self.users[i].identity, 10, visibility=1
|
||||||
|
)
|
||||||
|
|
||||||
|
# Get info for TV show - should include ratings from episode
|
||||||
|
tvshow_info = Rating.get_info_for_item(self.tvshow)
|
||||||
|
|
||||||
|
# Check count (1 for show + 5 for episode = 6)
|
||||||
|
self.assertEqual(tvshow_info["count"], 6)
|
||||||
|
|
||||||
|
# The average should consider all ratings (6 + 5*10 = 56, divided by 6 = 9.3)
|
||||||
|
self.assertEqual(tvshow_info["average"], 9.3)
|
||||||
|
|
@ -14,7 +14,7 @@ from django.views.decorators.http import require_http_methods
|
||||||
|
|
||||||
from catalog.models import *
|
from catalog.models import *
|
||||||
from common.utils import AuthedHttpRequest, get_uuid_or_404
|
from common.utils import AuthedHttpRequest, get_uuid_or_404
|
||||||
from journal.models.renderers import convert_leading_space_in_md, render_md
|
from journal.models.renderers import convert_leading_space_in_md, has_spoiler, render_md
|
||||||
from users.middlewares import activate_language_for_user
|
from users.middlewares import activate_language_for_user
|
||||||
from users.models.apidentity import APIdentity
|
from users.models.apidentity import APIdentity
|
||||||
|
|
||||||
|
|
@ -135,9 +135,12 @@ class ReviewFeed(Feed):
|
||||||
return reviews
|
return reviews
|
||||||
|
|
||||||
def item_title(self, item: Review):
|
def item_title(self, item: Review):
|
||||||
return _("{review_title} - a review of {item_title}").format(
|
s = _("{review_title} - a review of {item_title}").format(
|
||||||
review_title=item.title, item_title=item.item.title
|
review_title=item.title, item_title=item.item.title
|
||||||
)
|
)
|
||||||
|
if has_spoiler(item.body):
|
||||||
|
s += " (" + _("may contain spoiler or triggering content") + ")"
|
||||||
|
return s
|
||||||
|
|
||||||
def item_description(self, item: Review):
|
def item_description(self, item: Review):
|
||||||
target_html = (
|
target_html = (
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
55
mkdocs.yml
55
mkdocs.yml
|
|
@ -1,55 +0,0 @@
|
||||||
site_name: 🧩 NeoDB
|
|
||||||
site_url: https://neodb.net
|
|
||||||
repo_url: http://github.com/neodb-social/neodb
|
|
||||||
edit_uri: blob/main/docs/
|
|
||||||
site_description: Mark the things you love.
|
|
||||||
nav:
|
|
||||||
- Overview: 'index.md'
|
|
||||||
- features.md
|
|
||||||
- servers.md
|
|
||||||
- apps.md
|
|
||||||
- install.md
|
|
||||||
- upgrade.md
|
|
||||||
- configuration.md
|
|
||||||
- troubleshooting.md
|
|
||||||
- development.md
|
|
||||||
- api.md
|
|
||||||
- origin.md
|
|
||||||
theme:
|
|
||||||
logo: assets/logo.svg
|
|
||||||
name: material
|
|
||||||
font:
|
|
||||||
text: Lato
|
|
||||||
palette:
|
|
||||||
primary: custom
|
|
||||||
accent: custom
|
|
||||||
features:
|
|
||||||
# - navigation.tabs
|
|
||||||
# - navigation.footer
|
|
||||||
custom_dir: docs/templates
|
|
||||||
extra:
|
|
||||||
social:
|
|
||||||
- icon: fontawesome/brands/mastodon
|
|
||||||
link: https://mastodon.online/@neodb/
|
|
||||||
- icon: fontawesome/brands/bluesky
|
|
||||||
link: https://bsky.app/profile/neodb.net
|
|
||||||
- icon: fontawesome/brands/x-twitter
|
|
||||||
link: https://twitter.com/NeoDBsocial
|
|
||||||
- icon: fontawesome/brands/threads
|
|
||||||
link: https://www.threads.net/@neodb.social
|
|
||||||
- icon: fontawesome/brands/discord
|
|
||||||
link: https://discord.gg/QBHkrV8bxK
|
|
||||||
- icon: fontawesome/brands/github
|
|
||||||
link: http://github.com/neodb-social/neodb
|
|
||||||
- icon: fontawesome/brands/docker
|
|
||||||
link: https://hub.docker.com/u/neodb
|
|
||||||
extra_css:
|
|
||||||
- stylesheets/extra.css
|
|
||||||
markdown_extensions:
|
|
||||||
- pymdownx.superfences:
|
|
||||||
custom_fences:
|
|
||||||
- name: mermaid
|
|
||||||
class: mermaid
|
|
||||||
format: !!python/name:pymdownx.superfences.fence_code_format
|
|
||||||
hooks:
|
|
||||||
- mkdocs_hook.py
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
import json
|
|
||||||
|
|
||||||
|
|
||||||
def on_page_markdown(markdown, page, config, **kwargs):
|
|
||||||
if page.url == "servers/":
|
|
||||||
with open(config.docs_dir + "/servers.json") as f:
|
|
||||||
servers = json.load(f)
|
|
||||||
m = ""
|
|
||||||
for s in servers["servers"]:
|
|
||||||
host = s["host"]
|
|
||||||
name = s.get("name", host)
|
|
||||||
admin = s.get("admin", [])
|
|
||||||
label = s.get("label", [])
|
|
||||||
language = s.get("language", [])
|
|
||||||
description = s.get("description", "")
|
|
||||||
m += f" - **[{name}](https://{host})**"
|
|
||||||
if label:
|
|
||||||
m += f" {' '.join([f'`{a}`' for a in label])}"
|
|
||||||
if language:
|
|
||||||
m += f" {' '.join([f'`{a}`' for a in language])}"
|
|
||||||
if description:
|
|
||||||
m += f" \n {description}"
|
|
||||||
if admin:
|
|
||||||
m += f" \n admin: {', '.join([f'`{a}`' for a in admin])}"
|
|
||||||
m += "\n"
|
|
||||||
return markdown.replace("{servers}", m)
|
|
||||||
|
|
@ -9,7 +9,7 @@
|
||||||
# generate-hashes: false
|
# generate-hashes: false
|
||||||
# universal: false
|
# universal: false
|
||||||
|
|
||||||
aiohappyeyeballs==2.4.6
|
aiohappyeyeballs==2.5.0
|
||||||
# via aiohttp
|
# via aiohttp
|
||||||
aiohttp==3.11.13
|
aiohttp==3.11.13
|
||||||
# via discord-py
|
# via discord-py
|
||||||
|
|
@ -62,10 +62,10 @@ cssbeautifier==1.15.4
|
||||||
# via djlint
|
# via djlint
|
||||||
dateparser==1.2.1
|
dateparser==1.2.1
|
||||||
deepmerge==2.0
|
deepmerge==2.0
|
||||||
discord-py==2.5.0
|
discord-py==2.5.2
|
||||||
distlib==0.3.9
|
distlib==0.3.9
|
||||||
# via virtualenv
|
# via virtualenv
|
||||||
django==4.2.19
|
django==4.2.20
|
||||||
# via django-anymail
|
# via django-anymail
|
||||||
# via django-appconf
|
# via django-appconf
|
||||||
# via django-auditlog
|
# via django-auditlog
|
||||||
|
|
@ -130,7 +130,7 @@ httpcore==1.0.7
|
||||||
# via httpx
|
# via httpx
|
||||||
httpx==0.28.1
|
httpx==0.28.1
|
||||||
# via atproto
|
# via atproto
|
||||||
identify==2.6.8
|
identify==2.6.9
|
||||||
# via pre-commit
|
# via pre-commit
|
||||||
idna==3.10
|
idna==3.10
|
||||||
# via anyio
|
# via anyio
|
||||||
|
|
@ -138,7 +138,7 @@ idna==3.10
|
||||||
# via requests
|
# via requests
|
||||||
# via yarl
|
# via yarl
|
||||||
igdb-api-v4==0.3.3
|
igdb-api-v4==0.3.3
|
||||||
jinja2==3.1.5
|
jinja2==3.1.6
|
||||||
# via mkdocs
|
# via mkdocs
|
||||||
# via mkdocs-material
|
# via mkdocs-material
|
||||||
jsbeautifier==1.15.4
|
jsbeautifier==1.15.4
|
||||||
|
|
@ -159,7 +159,7 @@ markdown==3.7
|
||||||
# via mkdocs
|
# via mkdocs
|
||||||
# via mkdocs-material
|
# via mkdocs-material
|
||||||
# via pymdown-extensions
|
# via pymdown-extensions
|
||||||
markdownify==1.0.0
|
markdownify==1.1.0
|
||||||
markupsafe==3.0.2
|
markupsafe==3.0.2
|
||||||
# via jinja2
|
# via jinja2
|
||||||
# via mkdocs
|
# via mkdocs
|
||||||
|
|
@ -201,7 +201,7 @@ pre-commit==4.1.0
|
||||||
propcache==0.3.0
|
propcache==0.3.0
|
||||||
# via aiohttp
|
# via aiohttp
|
||||||
# via yarl
|
# via yarl
|
||||||
protobuf==5.29.3
|
protobuf==6.30.0
|
||||||
# via igdb-api-v4
|
# via igdb-api-v4
|
||||||
psycopg2-binary==2.9.10
|
psycopg2-binary==2.9.10
|
||||||
pycparser==2.22
|
pycparser==2.22
|
||||||
|
|
@ -252,7 +252,7 @@ rjsmin==1.2.2
|
||||||
# via django-compressor
|
# via django-compressor
|
||||||
rq==2.1.0
|
rq==2.1.0
|
||||||
# via django-rq
|
# via django-rq
|
||||||
ruff==0.9.9
|
ruff==0.9.10
|
||||||
sentry-sdk==2.22.0
|
sentry-sdk==2.22.0
|
||||||
setproctitle==1.3.5
|
setproctitle==1.3.5
|
||||||
six==1.17.0
|
six==1.17.0
|
||||||
|
|
@ -285,7 +285,7 @@ typing-extensions==4.12.2
|
||||||
# via pydantic
|
# via pydantic
|
||||||
# via pydantic-core
|
# via pydantic-core
|
||||||
# via pyright
|
# via pyright
|
||||||
tzlocal==5.3
|
tzlocal==5.3.1
|
||||||
# via dateparser
|
# via dateparser
|
||||||
urllib3==2.3.0
|
urllib3==2.3.0
|
||||||
# via django-anymail
|
# via django-anymail
|
||||||
|
|
@ -293,7 +293,7 @@ urllib3==2.3.0
|
||||||
# via sentry-sdk
|
# via sentry-sdk
|
||||||
urlman==2.0.2
|
urlman==2.0.2
|
||||||
validators==0.34.0
|
validators==0.34.0
|
||||||
virtualenv==20.29.2
|
virtualenv==20.29.3
|
||||||
# via pre-commit
|
# via pre-commit
|
||||||
watchdog==6.0.0
|
watchdog==6.0.0
|
||||||
# via mkdocs
|
# via mkdocs
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@
|
||||||
# generate-hashes: false
|
# generate-hashes: false
|
||||||
# universal: false
|
# universal: false
|
||||||
|
|
||||||
aiohappyeyeballs==2.4.6
|
aiohappyeyeballs==2.5.0
|
||||||
# via aiohttp
|
# via aiohttp
|
||||||
aiohttp==3.11.13
|
aiohttp==3.11.13
|
||||||
# via discord-py
|
# via discord-py
|
||||||
|
|
@ -48,8 +48,8 @@ cryptography==44.0.2
|
||||||
# via atproto
|
# via atproto
|
||||||
dateparser==1.2.1
|
dateparser==1.2.1
|
||||||
deepmerge==2.0
|
deepmerge==2.0
|
||||||
discord-py==2.5.0
|
discord-py==2.5.2
|
||||||
django==4.2.19
|
django==4.2.20
|
||||||
# via django-anymail
|
# via django-anymail
|
||||||
# via django-appconf
|
# via django-appconf
|
||||||
# via django-auditlog
|
# via django-auditlog
|
||||||
|
|
@ -116,7 +116,7 @@ loguru==0.7.3
|
||||||
lxml==5.3.1
|
lxml==5.3.1
|
||||||
markdown==3.7
|
markdown==3.7
|
||||||
# via django-markdownx
|
# via django-markdownx
|
||||||
markdownify==1.0.0
|
markdownify==1.1.0
|
||||||
mistune==3.1.2
|
mistune==3.1.2
|
||||||
multidict==6.1.0
|
multidict==6.1.0
|
||||||
# via aiohttp
|
# via aiohttp
|
||||||
|
|
@ -132,7 +132,7 @@ podcastparser==0.6.10
|
||||||
propcache==0.3.0
|
propcache==0.3.0
|
||||||
# via aiohttp
|
# via aiohttp
|
||||||
# via yarl
|
# via yarl
|
||||||
protobuf==5.29.3
|
protobuf==6.30.0
|
||||||
# via igdb-api-v4
|
# via igdb-api-v4
|
||||||
psycopg2-binary==2.9.10
|
psycopg2-binary==2.9.10
|
||||||
pycparser==2.22
|
pycparser==2.22
|
||||||
|
|
@ -190,7 +190,7 @@ typing-extensions==4.12.2
|
||||||
# via beautifulsoup4
|
# via beautifulsoup4
|
||||||
# via pydantic
|
# via pydantic
|
||||||
# via pydantic-core
|
# via pydantic-core
|
||||||
tzlocal==5.3
|
tzlocal==5.3.1
|
||||||
# via dateparser
|
# via dateparser
|
||||||
urllib3==2.3.0
|
urllib3==2.3.0
|
||||||
# via django-anymail
|
# via django-anymail
|
||||||
|
|
|
||||||
209
test_data/https___music_apple_com_cn_album_1284391545
Normal file
209
test_data/https___music_apple_com_cn_album_1284391545
Normal file
File diff suppressed because one or more lines are too long
201
test_data/https___music_apple_com_fr_album_1284391545
Normal file
201
test_data/https___music_apple_com_fr_album_1284391545
Normal file
File diff suppressed because one or more lines are too long
197
test_data/https___music_apple_com_jp_album_1284391545
Normal file
197
test_data/https___music_apple_com_jp_album_1284391545
Normal file
File diff suppressed because one or more lines are too long
197
test_data/https___music_apple_com_kr_album_1284391545
Normal file
197
test_data/https___music_apple_com_kr_album_1284391545
Normal file
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Loading…
Add table
Reference in a new issue