more doc

README.md

@@ -1,23 +1,18 @@
-# Boofilsic
-An application allows you to mark any books, movies and more things you love. 
+# Boofilsic/NeoDB
+Boofilsic/NeoDB is an open source project and free service to help users manage, share and discover collections, reviews and ratings for culture products (e.g. books, movies, music, podcasts, games and performances) in Fediverse. 
 
-Works with Mastodon API and Twitter API.  
+[NeoDB.social](https://neodb.social) and [NiceDB](https://nicedb.org) are free services hosted by volunteers.
 
 ## Install
-Please see [doc/GUIDE.md](doc/GUIDE.md)
+Please see [doc/install.md](doc/install.md)
 
 ## Bug Report
  - to file a bug for NiceDB, please create an issue [here](https://github.com/doubaniux/boofilsic/issues/new)
  - to file a bug or request new features for NeoDB, please contact NeoDB on [Fediverse](https://mastodon.social/@neodb) or [Twitter](https://twitter.com/NeoDBsocial)
 
 ## Contribution
-The project is based on Django. If you are familiar with this technique and willing to read through the terrible code😝, your contribution would be the most welcome!
-
-Currently looking for someone to help with:
-- Writing instrucations on how to deploy this
-- Explaining the structure of code
-- Refactoring (this is something big)
+ - Please see [doc/development.md](doc/development.md) for some basics to start with
+ - Join our Discord community, links available on [our Fediverse profile](https://mastodon.social/@neodb)
 
 ## Sponsor
 If you like this project, please consider sponsoring NiceDB on [Patreon](https://patreon.com/tertius).
-

doc/configuration.md

@@ -0,0 +1,24 @@
+Configuration
+=============
+
+
+Settings you may want to change
+-------------------------------
+most settings resides in `settings.py`, a few critical ones:
+
+ - `SECRET_KEY` back it up well somewhere
+ - `SITE_INFO['site_name']` change by you need
+ - `CLIENT_NAME` site now show up in Mastodon app page
+ - `REDIRECT_URIS` 
+ - `APP_WEBSITE` external root url for your side
+ - `REDIRECT_URIS` this should be `APP_WEBSITE + "/users/OAuth2_login/"` . It can be multiple urls separated by `\n` , but not all Fediverse software supports it well. Also note changing this later may invalidate app token granted previously
+ - `MASTODON_ALLOW_ANY_SITE` set to `True` so that user can login via any Mastodon API compatible sites (e.g. Mastodon/Pleroma)
+ - `MASTODON_CLIENT_SCOPE` change it later may invalidate app token granted previously
+ - `ADMIN_URL` admin page url, keep it private
+ - `SEARCH_BACKEND` should be either `TYPESENSE` or `MEILISEARCH` so that search and index can function. set as `None` will use default database search, which will be for development only and will not work very well in production
+ 
+
+Settings for Scrapers
+---------------------
+
+TBA

doc/development.md

@@ -3,7 +3,7 @@ Development
 
 *this doc is based on new data models work which is a work in progress*
 
-First, a working version of local NeoDB instance has to be established, see [install guide](GUIDE.md).
+First, a working version of local NeoDB instance has to be established, see [install guide](install.md).
 
 Since new data model is still under development, most pieces are off by default, add `new_data_model=1` to your shell env and run migrations before start working on these new models
 
@@ -36,9 +36,9 @@ Preserving test database for alias 'default'...
 ```
 
 
-Data Models
------------
-main django apps for NeoDB:
+Applications
+------------
+Main django apps for NeoDB:
  - `users` manages user in typical django fashion
  - `mastodon` this leverages [Mastodon API](https://docs.joinmastodon.org/client/intro/) and [Twitter API](https://developer.twitter.com/en/docs/twitter-api) for user login and data sync
  - `catalog` manages different types of items user may review, and scrapers to fetch from external resources, see [catalog.md](catalog.md) for more details
@@ -46,19 +46,3 @@ main django apps for NeoDB:
  - `social` manages timeline for local users and ActivityStreams for remote servers, see [social.md](social.md) for more details
 
 These apps are legacy: books, music, movies, games, collections, they will be removed soon.
-
-
-ActivityPub
------------
-
-TBA
-
-References:
- - https://www.w3.org/TR/activitypub/
- - https://www.w3.org/TR/activitystreams-core/
- - https://www.w3.org/TR/activitystreams-vocabulary/
- - https://www.w3.org/TR/json-ld/
- - https://codeberg.org/fediverse/fep/src/branch/main/feps/fep-e232.md
- - https://socialhub.activitypub.rocks/t/guide-for-new-activitypub-implementers/479
- - https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/
- - https://docs.joinmastodon.org/spec/activitypub/

doc/install.md

@@ -20,6 +20,7 @@ Create and edit your own configuration file (optional but very much recommended)
 mkdir mysite && cp boofilsic/settings.py mysite/
 export DJANGO_SETTINGS_MODULE=mysite.settings
 ```
+More details on `settings.py` in [configuration.md](configuration.md)
 
 Create and use `venv` as you normally would, then install packages 
 ```

doc/journal.md

@@ -0,0 +1,98 @@
+Journal
+=======
+
+
+Data Model
+----------
+```mermaid
+classDiagram
+User .. Piece
+
+class Piece {
+    +User owner
+    +int visibility
+}
+
+class Item {
+    +str title
+    +str brief
+    -enum type
+}
+Piece <|-- Content
+Item .. Content
+class Content {
+    +Item target
+}
+Content <|-- Rating 
+class Rating {
+    +int grade
+}
+Content <|-- Review 
+class Review {
+    +Enum warning_type
+    +str title
+    +str text
+}
+
+Content <|-- Note
+class Note {
+    +str text
+    +int position
+    +enum position_type
+    +str quotation
+    +Image image 
+    
+}
+Content <|-- Reply
+class Reply {
+    +Content reply_to
+}
+Piece <|-- List
+Item <|-- List
+class List{
+    +ListItem[] items
+}
+Item .. ListItem
+List *-- ListItem
+class ListItem {
+    +Item item
+    +Comment comment
+    +Dict metadata
+}
+List <|-- Collection 
+class Collection {
+    +str title
+    +str brief
+    +Bool collabrative
+}
+List <|-- Tag 
+class Tag {
+    +str title
+}
+List <|-- Shelf 
+class Shelf {
+    +Enum type
+}
+User .. ShelfLogManager
+class ShelfLogManager {
+    +User owner
+    +ShelfLogEntry[] logs
+}
+ShelfLogManager *-- ShelfLogEntry
+class ShelfLogEntry {
+    +Item item
+    +Shelf shelf
+    +DateTime timestamp
+}
+ShelfLogEntry .. Item
+ShelfLogEntry .. Shelf
+
+Shelf *-- ShelfItem
+ListItem <|-- ShelfItem
+
+ListItem <|-- TagItem
+ListItem <|-- CollectionItem
+
+Tag *-- TagItem
+Collection *-- CollectionItem
+```

doc/social.md

@@ -0,0 +1,59 @@
+Social
+======
+
+Data Modal
+----------
+```mermaid
+classDiagram
+User .. Piece
+
+class Piece {
+    +User owner
+    +int visibility
+}
+User .. Activity
+class Activity {
+    +User owner
+    +int visibility
+    +Enum action_type
+    +Piece action_object
+    +Item target
+    +Bool is_viewable
+}
+Activity .. Piece
+Activity .. Item
+class Item {
+    +str title
+    +str brief
+    -enum type
+}
+```
+
+Activities
+----------
+Activity is used for two main purposes:
+ - Generate chronological view for users
+ - Generate ActivityStreams OrderedCollection for ActivityPub interactions
+
+
+
+
+
+ActivityPub
+-----------
+
+TBA
+
+References:
+ - https://www.w3.org/TR/activitypub/
+ - https://www.w3.org/TR/activitystreams-core/
+ - https://www.w3.org/TR/activitystreams-vocabulary/
+ - https://www.w3.org/TR/json-ld/
+ - https://codeberg.org/fediverse/fep/src/branch/main/feps/fep-e232.md
+ - https://socialhub.activitypub.rocks/t/guide-for-new-activitypub-implementers/479
+ - https://docs.joinmastodon.org/spec/activitypub/
+ - https://docs.joinbookwyrm.com/activitypub.html
+ - https://github.com/Podcastindex-org/podcast-namespace/blob/main/proposal-docs/social/social.md#socialinteract-element
+ - https://dev.funkwhale.audio/funkwhale/funkwhale/-/tree/develop/docs/developer_documentation/federation
+ - https://github.com/inventaire/inventaire/issues/187
+ - https://github.com/inventaire/inventaire/issues/533

misc/dev-reset.sh

@@ -3,9 +3,9 @@
 
 [ -f manage.py ] || exit $1
 
-echo "\033[0;31mWARNING"
+echo "\033[0;31mWARNING: this script will destroy all neodb databases and migrations"
 while true; do
-    read -p "Do you wish to continue destroy all databases and migrations? (yes/no) " yn
+    read -p "Do you wish to continue? (yes/no) " yn
     case $yn in
         [Yy]* ) break;;
         [Nn]* ) exit;;
@@ -22,7 +22,7 @@ psql $* neodb -c "CREATE EXTENSION hstore WITH SCHEMA public;" || exit $?
 
 find -type d -name migrations | xargs rm -rf
 
-python3 manage.py makemigrations mastodon users books movies games music sync management collection catalog journal social || exit $?
+python3 manage.py makemigrations mastodon users books movies games music sync management collection common sync management catalog journal social || exit $?
 
 python3 manage.py migrate || exit $?
 

social/models.py

@@ -23,6 +23,8 @@ class ActionType(models.TextChoices):
     Create = 'create'
     Delete = 'delete'
     Update = 'update'
+    Add = 'add'
+    Remove = 'remove'
     Like = 'like'
     Undo_Like = 'undo_like'
     Announce = 'announce'

social/tests.py

@@ -35,7 +35,7 @@ class SocialTest(TestCase):
         timeline = self.alice.activity_manager.get_viewable_activities()
         self.assertEqual(len(timeline), 2)
 
-        # bon see 0 activity in timeline in the beginning
+        # bob see 0 activity in timeline in the beginning
         timeline2 = self.bob.activity_manager.get_viewable_activities()
         self.assertEqual(len(timeline2), 0)