Yohan Chalier Projects Portfolio Blog

Personnal Notes Management System


Orgapy

As stated in the Notifpy project, I want to get rid of my Google account, without loosing its features. This Django application replaces (and goes beyond) the Google Keep application.

Notes

The main purpose of Keep being taking notes, this is the first objective of my application. Originally, it was not even a Django application, but an interface for reading and writing small text files directly on the server. The system worked rather fine but lacked in lots of features, which this new application tries to make up for.

Markdown Markup

Notes are stored in a SQL database, as text content enriched with Markdown markup. Such markup is easy to use and type for the writer, and provides a good coverage of the possible styles one may want to use for note keeping. The translation from Markdown to HTML is handled by Mistune, a Python module. As specified in its documentation, I wrote a custom parser for Mistune that adds two more features to the default version:

  • highlight code syntax with the Pygments module,
  • and handle checkboxes parsing.

Pygments module tags words within a block of code with classes offering style customization possibilities through CSS. Dark theme preferences lead me to choose the Monokai theme for that matter.

Checkboxes are a feature available in the GitHub Markdown parser, but is not natively supported by Mistune. The syntax is the following:

- [x] Checked element
- [ ] Unchecked element
- [ ] Another unchecked element

My custom parser checks for such patterns in already detected list items, and inserts HTML <input type="checkbox" /> tags when suited. Those tags are given an integer id by the parser so they can be uniquely identified when the player clicks on them, allowing for dynamic checking directly from the note reading view. The final parser is the following:

class HighlightRenderer(mistune.HTMLRenderer):
    """Custom mistune renderder to handle syntax highlighting"""

    def __init__(self, *args, **kwargs):
        mistune.HTMLRenderer.__init__(self, *args, **kwargs)
        self.checkboxes_index = 0

    def block_code(self, code, lang=None):
        """Use pygments to highlight a markdown code block"""
        if lang:
            try:
                lexer = get_lexer_by_name(lang, stripall=True)
                formatter = html.HtmlFormatter()
                return highlight(code, lexer, formatter)
            except ClassNotFound:
                return "<pre><code>" + mistune.escape(code) + "</code></pre>"
        return "<pre><code>" + mistune.escape(code) + "</code></pre>"

    def list_item(self, text, level):
        """Render list item with task list support"""
        old_list_item = mistune.HTMLRenderer.list_item
        new_list_item = lambda _, text: "<li class=\"task-list-item\">%s</li>\n" % text
        task_list_re = re.compile(r"\[[xX ]\] ")
        match = task_list_re.match(text)
        if match is None:
            return old_list_item(self, text, level)
        prefix = match.group()
        checked = False
        checkbox_id = self.checkboxes_index
        self.checkboxes_index += 1
        if prefix[1].lower() == "x":
            checked = True
        checked_txt = ""
        if checked:
            checked_txt = "checked"
        template = """
        <input type="checkbox" class="task-list-item-checkbox" id="cb{cid}" {checked}/>
        <label for="cb{cid}">{content}</label>
        """.strip()
        return new_list_item(self, template.format(
            cid=checkbox_id,
            checked=checked_txt,
            content=text[match.end():]
        ))

One last feature offered by such markup is the export as PDF of each note. Several options were available. I first used Pandoc, but it was too heavy for my small Raspberry Pi. Instead, I use a Python module called xhtml2pdf, taking as input the HTML produced by Mistune. This allows for the use of my Mistune custom parser also in the PDF generation.

Public Access

Notes are by default only accessible to the Django user that created them. However, a flag can be set so that the note becomes accessible by anyone having the link to that note.

Such public notes can be registered in a "blog", i.e. listed on a public view, then acting as blog posts. Publications are accompanied by a date of publication and an author name. No commenting system has been implemented so far, but that is candidate for future work.

Tasks and Deadlines

Another feature of Google Keep is the addition of deadlines to notes, for remembering their author at some point. Orgapy replicates that feature by expanding the regular note model with a due_date field, and separating, in the view, those tasks from the other notes. Specific styling is applied regarding the distance in time between the due date and the current date, putting the emphasis on close (or missed) deadlines.

Book Quotes

One last expansion of regular notes is the book quote model. I like to write down some sentences I find interesting while reading, and used to do that in a single text document, not making it easy for information retrieval once the file gets longer. To enhance that, I added two models: one for authors and one for works. A book quote is then a note with a reference to an author's work. Quotes can then be grouped by author and by work, which greatly helps their organization.

Regular Objectives

After some time using the application, I felt the need to add another feature: daily and weekly objectives follow-up. This is something that is regularly done in bullet journals. Keeping track of particulars tasks that must be done each day or each weeks helps developping habits that one may not find the appropriate motivation for otherwise.

The goal was to implement them with a nice visualization on the client side. My idea was to mimic GitHub's contributions grid. Here is how it looks:

Internally, this information is stored as a bit string with a date offset, the i-th bit being set at 1 if the task has been performed at the i-th time period (either a day for a daily objective, or a week for a weekly objective). The string is updated and padded if necessary whenever a task is marked done.

Calendar Integration

The last feature I added was the integration of a CalDAV client, communicating with my self-hosted Radicale server. For simplicity, no event creation or update features are implemented. Only incoming events are listed.