Commit graph

600 commits

Author SHA1 Message Date
Your Name
9802775994 before adding some real tests 2022-11-10 14:17:48 -05:00
Henri Dickson
5da627a8df
add all NeoDB features to NiceDB (#115)
* fix scraping failure with wepb image (merge upstream/fix-webp-scrape)

* add filetype to requirements

* add proxycrawl.com as fallback for douban scraper

* load 3p js/css from cdn

* add fix-cover task

* fix book/album cover tasks

* scrapestack

* bandcamp scrape and preview ;
manage.py scrape <url> ;
make ^C work when DEBUG

* use scrapestack when fix cover

* add user agent to improve compatibility

* search BandCamp for music albums

* add missing MovieGenre

* fix search 500 when song has no parent album

* adjust timeout

* individual scrapers

* fix tmdb parser

* export marks via rq; pref to send public toot; move import to data page

* fix spotify import

* fix edge cases

* export: fix dupe tags

* use rq to manage doufen import

* add django command to manage rq jobs

* fix export edge case

* tune rq admin

* fix detail page 502 step 1: async pull mastodon follow/block/mute list

* fix detail page 502 step 2: calculate relationship by local cached data

* manual sync mastodon follow info

* domain_blocks parsing fix

* marks by who i follows

* adjust label

* use username in urls

* add page to list a user\'s review

* review widget on user home page

* fix preview 500

* fix typo

* minor fix

* fix google books parsing

* allow mark/review visible to oneself

* fix auto sync masto for new user

* fix search 500

* add command to restart a sync task

* reset visibility

* delete user data

* fix tag search result pagination

* not upgrade to django 4 yet

* basic doc

* wip: collection

* wip

* wip

* collection use htmx

* show in-collection section for entities

* fix typo

* add su for easier debug

* fix some 500s

* fix login using alternative domain

* hide data from disabled user

* add item to list from detail page

* my tags

* collection: inline comment edit

* show number of ratings

* fix collection delete

* more detail in collection view

* use item template in search result

* fix 500

* write index to meilisearch

* fix search

* reindex in batch

* fix 500

* show search result from meilisearch

* more search commands

* index less fields

* index new items only

* search highlights

* fix 500

* auto set search category

* classic search if no meili server

* fix index stats error

* support typesense backend

* workaround typesense bug

* make external search async

* fix 500, typo

* fix cover scripts

* fix minor issue in douban parser

* supports m.douban.com and customized bandcamp domain

* move account

* reword with gender-friendly and instance-neutral language

* Friendica does not have vapid_key in api response

* enable anonymous search

* tweak book result template

* API v0

API v0

* fix meilisearch reindex

* fix search by url error

* login via twitter.com

* login via pixelfed

* minor fix

* no refresh on inactive users

* support refresh access token

* get rid of /users/number-id/

* refresh twitter handler automatically

* paste image when review

* support PixelFed (very long token)

* fix django-markdownx version

* ignore single quote for meilisearch for now

* update logo

* show book review/mark from same isbn

* show movie review/mark from same imdb

* fix login with older mastodon servers

* import Goodreads book list and profile

* add timestamp to Goodreads import

* support new google books api

* import goodreads list

* minor goodreads fix

* click corner action icon to add to wishlist

* clean up duplicated code

* fix anonymous search

* fix 500

* minor fix search 500

* show rating only if votes > 5

* Entity.refresh_rating()

* preference to append text when sharing; clean up duplicated code

* fix missing data for user tagged view

* fix page link for tag view

* fix 500 when language field longer than 10

* fix 500 when sharing mark for song

* fix error when reimport goodread profile

* fix minor typo

* fix a rare 500

* error log dump less

* fix tags in marks export

* fix missing param in pagination

* import douban review

* clarify text

* fix missing sheet in review import

* review: show in progress

* scrape douban: ignore unknown genre

* minor fix

* improve review import by guess entity urls

* clear guide text for review import

* improve review import form text

* workaround some 500

* fix mark import error

* fix img in review import

* load external results earlier

* ignore search server errors

* simplify user register flow to avoid inconsistent state

* Add a learn more link on login page

* Update login.html

* show mark created timestamp as mark time

* no 500 for api error

* redirect for expired tokens

* ensure preference object created.

* mark collections

* tag list

* fix tag display

* fix sorting etc

* fix 500

* fix potential export 500; save shared links

* fix share to twittwe

* fix review url

* fix 500

* fix 500

* add timeline, etc

* missing status change in timeline

* missing id in timeline

* timeline view by default

* workaround bug in markdownx...

* fix typo

* option to create new collection when add from detail page

* add missing announcement and tags in timeline home

* add missing announcement

* add missing announcement

* opensearch

* show fediverse shared link

* public review no longer requires login

* fix markdownx bug

* fix 500

* use cloudflare cdn

* validate jquery load and domain input

* fix 500

* tips for goodreads import

* collaborative collection

* show timeline and profile link on nav bar

* minor tweak

* share collection

* fix Goodreads search

* show wish mark in timeline

* resync failed urls with local proxy

* resync failed urls with local proxy: check proxy first

* scraper minor fix

* resync failed urls

* fix fields limit

* fix douban parsing error

* resync

* scraper minor fix

* scraper minor fix

* scraper minor fix

* local proxy

* local proxy

* sync default config from neodb

* configurable site name

* fix 500

* fix 500 for anonymous user

* add sentry

* add git version in log

* add git version in log

* no longer rely on cdnjs.cloudflare.com

* move jq/cash to _common_libs template partial

* fix rare js error

* fix 500

* avoid double submission error

* import tag in lower case

* catch some js network errors

* catch some js network errors

* support more goodread urls

* fix unaired tv in tmdb

* support more google book urls

* fix related series

* more goodreads urls

* robust googlebooks search

* robust search

* Update settings.py

* Update scraper.py

* Update requirements.txt

* make nicedb work

* doc update

* simplify permission check

* update doc

* update doc for bug report link

* skip spotify tracks

* fix 500

* improve search api

* blind fix import compatibility

* show years for movie in timeline

* show years for movie in timeline; thinner font

* export reviews

* revert user home to use jquery https://github.com/fabiospampinato/cash/issues/246

* IGDB

* use IGDB for Steam

* use TMDB for IMDb

* steam: igdb then fallback to steam

* keep change history

* keep change history: add django settings

* Steam: keep localized title/brief while merging IGDB

* basic Docker support

* rescrape

* Create codeql-analysis.yml

* Create SECURITY.md

* Create pysa.yml

Co-authored-by: doubaniux <goodsir@vivaldi.net>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Their Name <they@example.com>
Co-authored-by: Mt. Front <mfcndw@gmail.com>
2022-11-09 19:56:50 +01:00
Henri Dickson
14b003a44a add all NeoDB features to NiceDB (#115)
* fix scraping failure with wepb image (merge upstream/fix-webp-scrape)

* add filetype to requirements

* add proxycrawl.com as fallback for douban scraper

* load 3p js/css from cdn

* add fix-cover task

* fix book/album cover tasks

* scrapestack

* bandcamp scrape and preview ;
manage.py scrape <url> ;
make ^C work when DEBUG

* use scrapestack when fix cover

* add user agent to improve compatibility

* search BandCamp for music albums

* add missing MovieGenre

* fix search 500 when song has no parent album

* adjust timeout

* individual scrapers

* fix tmdb parser

* export marks via rq; pref to send public toot; move import to data page

* fix spotify import

* fix edge cases

* export: fix dupe tags

* use rq to manage doufen import

* add django command to manage rq jobs

* fix export edge case

* tune rq admin

* fix detail page 502 step 1: async pull mastodon follow/block/mute list

* fix detail page 502 step 2: calculate relationship by local cached data

* manual sync mastodon follow info

* domain_blocks parsing fix

* marks by who i follows

* adjust label

* use username in urls

* add page to list a user\'s review

* review widget on user home page

* fix preview 500

* fix typo

* minor fix

* fix google books parsing

* allow mark/review visible to oneself

* fix auto sync masto for new user

* fix search 500

* add command to restart a sync task

* reset visibility

* delete user data

* fix tag search result pagination

* not upgrade to django 4 yet

* basic doc

* wip: collection

* wip

* wip

* collection use htmx

* show in-collection section for entities

* fix typo

* add su for easier debug

* fix some 500s

* fix login using alternative domain

* hide data from disabled user

* add item to list from detail page

* my tags

* collection: inline comment edit

* show number of ratings

* fix collection delete

* more detail in collection view

* use item template in search result

* fix 500

* write index to meilisearch

* fix search

* reindex in batch

* fix 500

* show search result from meilisearch

* more search commands

* index less fields

* index new items only

* search highlights

* fix 500

* auto set search category

* classic search if no meili server

* fix index stats error

* support typesense backend

* workaround typesense bug

* make external search async

* fix 500, typo

* fix cover scripts

* fix minor issue in douban parser

* supports m.douban.com and customized bandcamp domain

* move account

* reword with gender-friendly and instance-neutral language

* Friendica does not have vapid_key in api response

* enable anonymous search

* tweak book result template

* API v0

API v0

* fix meilisearch reindex

* fix search by url error

* login via twitter.com

* login via pixelfed

* minor fix

* no refresh on inactive users

* support refresh access token

* get rid of /users/number-id/

* refresh twitter handler automatically

* paste image when review

* support PixelFed (very long token)

* fix django-markdownx version

* ignore single quote for meilisearch for now

* update logo

* show book review/mark from same isbn

* show movie review/mark from same imdb

* fix login with older mastodon servers

* import Goodreads book list and profile

* add timestamp to Goodreads import

* support new google books api

* import goodreads list

* minor goodreads fix

* click corner action icon to add to wishlist

* clean up duplicated code

* fix anonymous search

* fix 500

* minor fix search 500

* show rating only if votes > 5

* Entity.refresh_rating()

* preference to append text when sharing; clean up duplicated code

* fix missing data for user tagged view

* fix page link for tag view

* fix 500 when language field longer than 10

* fix 500 when sharing mark for song

* fix error when reimport goodread profile

* fix minor typo

* fix a rare 500

* error log dump less

* fix tags in marks export

* fix missing param in pagination

* import douban review

* clarify text

* fix missing sheet in review import

* review: show in progress

* scrape douban: ignore unknown genre

* minor fix

* improve review import by guess entity urls

* clear guide text for review import

* improve review import form text

* workaround some 500

* fix mark import error

* fix img in review import

* load external results earlier

* ignore search server errors

* simplify user register flow to avoid inconsistent state

* Add a learn more link on login page

* Update login.html

* show mark created timestamp as mark time

* no 500 for api error

* redirect for expired tokens

* ensure preference object created.

* mark collections

* tag list

* fix tag display

* fix sorting etc

* fix 500

* fix potential export 500; save shared links

* fix share to twittwe

* fix review url

* fix 500

* fix 500

* add timeline, etc

* missing status change in timeline

* missing id in timeline

* timeline view by default

* workaround bug in markdownx...

* fix typo

* option to create new collection when add from detail page

* add missing announcement and tags in timeline home

* add missing announcement

* add missing announcement

* opensearch

* show fediverse shared link

* public review no longer requires login

* fix markdownx bug

* fix 500

* use cloudflare cdn

* validate jquery load and domain input

* fix 500

* tips for goodreads import

* collaborative collection

* show timeline and profile link on nav bar

* minor tweak

* share collection

* fix Goodreads search

* show wish mark in timeline

* resync failed urls with local proxy

* resync failed urls with local proxy: check proxy first

* scraper minor fix

* resync failed urls

* fix fields limit

* fix douban parsing error

* resync

* scraper minor fix

* scraper minor fix

* scraper minor fix

* local proxy

* local proxy

* sync default config from neodb

* configurable site name

* fix 500

* fix 500 for anonymous user

* add sentry

* add git version in log

* add git version in log

* no longer rely on cdnjs.cloudflare.com

* move jq/cash to _common_libs template partial

* fix rare js error

* fix 500

* avoid double submission error

* import tag in lower case

* catch some js network errors

* catch some js network errors

* support more goodread urls

* fix unaired tv in tmdb

* support more google book urls

* fix related series

* more goodreads urls

* robust googlebooks search

* robust search

* Update settings.py

* Update scraper.py

* Update requirements.txt

* make nicedb work

* doc update

* simplify permission check

* update doc

* update doc for bug report link

* skip spotify tracks

* fix 500

* improve search api

* blind fix import compatibility

* show years for movie in timeline

* show years for movie in timeline; thinner font

* export reviews

* revert user home to use jquery https://github.com/fabiospampinato/cash/issues/246

* IGDB

* use IGDB for Steam

* use TMDB for IMDb

* steam: igdb then fallback to steam

* keep change history

* keep change history: add django settings

* Steam: keep localized title/brief while merging IGDB

* basic Docker support

* rescrape

* Create codeql-analysis.yml

* Create SECURITY.md

* Create pysa.yml

Co-authored-by: doubaniux <goodsir@vivaldi.net>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Their Name <they@example.com>
Co-authored-by: Mt. Front <mfcndw@gmail.com>
2022-11-09 19:56:50 +01:00
Te Llamas
6acd109eca collection: show follows; order by follows 2022-11-09 02:41:53 +00:00
Te Llamas
f48252c19e add more fields to review feed 2022-11-08 03:12:50 +00:00
Te Llamas
1c7b294f9f minor fix 2022-11-08 02:37:15 +00:00
Te Llamas
204569ade5 add rss feed for reviews 2022-11-08 02:32:35 +00:00
Te Llamas
52b720cb73 reduce duplicated search results 2022-11-07 04:30:21 +00:00
Te Llamas
4eec1c1c57 try update existing mastodon status 2022-11-06 01:06:31 +00:00
Te Llamas
7329638567 add IGDB token refresh in configuration 2022-11-02 06:23:41 +00:00
Te Llamas
0aca28304d minor doc fix 2022-11-02 04:59:01 +00:00
Henri Dickson
8f7a2f6f40
Create pysa.yml 2022-09-22 16:42:45 -04:00
Henri Dickson
84449e5147
Create SECURITY.md 2022-09-22 16:31:32 -04:00
Henri Dickson
aa311eb6db
Create codeql-analysis.yml 2022-09-22 16:27:14 -04:00
Your Name
7cb4a376ce rescrape 2022-09-17 23:39:08 -04:00
Your Name
8fa1ba05a7 basic Docker support 2022-09-13 15:47:35 -04:00
Your Name
deeb684ec0 Steam: keep localized title/brief while merging IGDB 2022-09-12 11:24:25 -04:00
Your Name
7af717d0ba keep change history: add django settings 2022-09-08 19:46:48 -04:00
Your Name
4b81421419 keep change history 2022-09-08 19:41:56 -04:00
Your Name
9ca25169d3 steam: igdb then fallback to steam 2022-09-08 17:28:52 -04:00
Your Name
574f88f552 use TMDB for IMDb 2022-09-07 10:05:56 -04:00
Your Name
2be843a187 use IGDB for Steam 2022-09-07 09:44:08 -04:00
Your Name
0f03dfa785 IGDB 2022-09-07 00:35:11 -04:00
Your Name
4adb204ea1 revert user home to use jquery https://github.com/fabiospampinato/cash/issues/246 2022-08-20 13:20:18 -04:00
Your Name
01db91e610 export reviews 2022-08-20 00:14:18 -04:00
Your Name
81c793b18e show years for movie in timeline; thinner font 2022-08-19 22:15:49 -04:00
Your Name
0e50020fc7 show years for movie in timeline 2022-08-19 15:49:55 -04:00
Your Name
b481635203 blind fix import compatibility 2022-08-16 22:18:45 -04:00
Their Name
f5836b5437 improve search api 2022-08-10 01:53:27 +00:00
Their Name
6ee577620a fix 500 2022-08-09 15:50:18 +00:00
Your Name
f1f50b3000 skip spotify tracks 2022-08-05 14:23:51 -04:00
Your Name
fb46c6cd0c update doc for bug report link 2022-08-05 14:23:24 -04:00
Your Name
b46dc01dcb update doc 2022-07-31 21:02:03 -04:00
Your Name
14cc3ab431 simplify permission check 2022-07-31 20:58:19 -04:00
Your Name
96eb3a05f6 doc update 2022-07-31 03:07:11 -04:00
Your Name
e4e150b6b1 make nicedb work 2022-07-31 02:37:59 -04:00
alphatownsman
d4c1a00f15
Update requirements.txt 2022-07-29 21:50:53 -04:00
alphatownsman
d991afdd85
Update scraper.py 2022-07-29 21:40:04 -04:00
alphatownsman
2593c2c584
Update settings.py 2022-07-29 21:30:59 -04:00
alphatownsman
53f53ec6fd
Merge branch 'master' into neo 2022-07-29 21:27:20 -04:00
doubaniux
d4015a62d4
fix tag search result pagination (#109) (#114)
* fix imdb link parsing for DoubanMovie

* delete douban image download mixin

* add readme

* remove usage of eval

* fix tag search result pagination

Co-authored-by: Your Name <you@example.com>
Co-authored-by: doubaniux <56670055+doubaniux@users.noreply.github.com>
Co-authored-by: doubaniux <goodsir@vivaldi.net>

Co-authored-by: alphatownsman <90480431+alphatownsman@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
2022-07-27 21:58:25 +02:00
doubaniux
6c3e377bbe fix tag search result pagination (#109) (#114)
* fix imdb link parsing for DoubanMovie

* delete douban image download mixin

* add readme

* remove usage of eval

* fix tag search result pagination

Co-authored-by: Your Name <you@example.com>
Co-authored-by: doubaniux <56670055+doubaniux@users.noreply.github.com>
Co-authored-by: doubaniux <goodsir@vivaldi.net>

Co-authored-by: alphatownsman <90480431+alphatownsman@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
2022-07-27 21:58:25 +02:00
Your Name
738661c557 robust search 2022-07-23 14:45:47 -04:00
Your Name
7415b11f78 robust googlebooks search 2022-07-23 14:34:27 -04:00
Your Name
5a34198363 more goodreads urls 2022-07-23 14:09:12 -04:00
Your Name
d40f113c56 fix related series 2022-07-22 21:47:14 -04:00
Your Name
03fe53cec9 support more google book urls 2022-07-22 21:30:37 -04:00
Your Name
5fd6c696d4 fix unaired tv in tmdb 2022-07-22 21:16:52 -04:00
Your Name
f11102ef47 support more goodread urls 2022-07-22 21:10:58 -04:00
Your Name
9182157597 catch some js network errors 2022-07-21 09:50:36 -04:00