first try on database schema

don't log Mastodon 502 errors.

README.md

@@ -32,13 +32,15 @@ Today, you can use a Twitter, Mastodon, Telegram, and Mail with the account.
 They will communicate with each other; if someone warns others via Mail,
 Telegram, Twitter and Mastodon users will also see the message. And vice versa.
 
-In version 2, this repository contains a web application. On this website,
+In version 3, this repository contains a web application. On this website,
 people can register an own bot for their city - the website manages multiple
-bots for multiple citys. This way, you do not have to host it yourself.
+bots for multiple citys, which run in parallel. This way, you do not have to
+host it yourself, if you lack the know-how. But it is easily possible to do so
+with docker: http://github.com/ticketfrei/docker-ticketfrei/
 
-In the promotion folder, you'll find some promotion material you can use to
-build up such a community in your city. Unfortunately it is in german - but
-it's editable, feel free to translate it!
+In https://github.com/ticketfrei/promotion, you'll find some promotion material
+you can use to build up such a community in your city. Unfortunately it is in
+german - but it's editable, feel free to translate it!
 
 Website (our flagship instance): https://ticketfrei.links-tech.org
 
@@ -108,7 +110,7 @@ virtualenv -p python3 .
 Install the dependencies:
 
 ```shell
-pip install tweepy pytoml Mastodon.py bottle pyjwt pylibscrypt Markdown twx gitpython
+pip install tweepy pytoml Mastodon.py bottle pyjwt pylibscrypt Markdown twx gitpython SQLAlchemy
 ```
 
 Configure the bot:
@@ -152,19 +154,10 @@ echo "Enter your domain name into the following prompt:" && read DOMAIN
 # configure nginx
 sudo sed -r "s/example.org/$DOMAIN/g" deployment/example.org.conf > /etc/nginx/sites-enabled/$DOMAIN.conf
 
-# create folder for database
+# create folder for socket & database
 sudo mkdir /var/ticketfrei
 sudo chown www-data:www-data -R /var/ticketfrei
 
-# create folder for socket
-sudo mkdir /var/run/ticketfrei
-sudo chown -R www-data:www-data /var/run/ticketfrei
-sudo -s
-echo "mkdir /var/run/ticketfrei" >> /etc/rc.local
-echo "chown -R www-data:www-data /var/run/ticketfrei" >> /etc/rc.local
-echo "service ticketfrei-web restart" >> /etc/rc.local
-exit
-
 # change /etc/aliases permissions to be able to receive reports per mail
 sudo chown root:www-data /etc/aliases
 sudo chmod 664 /etc/aliases
@@ -187,17 +180,6 @@ sudo systemctl daemon-reload
 sudo systemctl start ticketfrei-backend.service
 ```
 
-### Backup
-
-For automated backups, you need to backup these files:
-
-* `/var/ticketfrei/db.sqlite`
-* `/srv/ticketfrei/config.toml`
-* `/etc/aliases`
-
-You can find an example how to do this with borgbackup in the deployment
-folder. Adjust it to your needs.
-
 ### Logs
 
 There are several logfiles which you can look at:
@@ -267,113 +249,3 @@ sudo chown $USER:$USER -R /var/log/ticketfrei
 ./frontend.py & ./backend.py &
 ```
 
-# Project History
-
-## Version 1
-
-- more of less hacked together during a mate-fueled weekend
-- backend-only, twitter & mastodon
-- just a script, which crawled & retweeted tweets, if they match a whitelist & blocklist
-- whitelist & blocklist were just 2 files
-
-## Version 2
-
-Reasons for the rewrite:
-- user management: Users should be able to run a Ticketfrei bot in their city
-  - without needing a server, without needing command line skills
-- more networks; not only Twitter & Mastodon, also Email & Telegram
-
-2 processes: backend & frontend.  
-The two Processes talk via a database.
-The two Processes have separate log files.
-Both processes take some config values from config.toml.
-
-### Backend
-
-The Backend takes care of crawling & spreading the reports.
-
-backend.py:
-- main loop which does the crawling & posting. 
-- loops through all cities in the database
-  - per city it tries all of the networks/bots:
-    - per network/bot it runs the crawl()-function to ask the social network for new reports
-    - then it checks whether the report is appropriate
-    - if yes, it posts the report via all networks/bots, which belong to the city.
-
-config.py: imports config values
-- Imports values from config.toml
-- If there is no config file it tries to use environment variables,
-- Apart from that it uses the default values.
-
-bot.py: bot parent Class
-- just the absolute minimum what a bot needs to be able to do: crawl + post
-- is never instantiated, only inherited from
-
-report.py: report Class
-- defines how reports are supposed to look like
-
-active_bots/mailbot.py as an example for how a network/bot works
-- crawl():
-  - mails arrive at an mbox file through exim4
-  - the bot checks whether they are new
-  - the bot generates a report object from the mail and returns it to the backend.py-loop
-- post():
-  - asks the database for the list of mails which want to receive reports for this city
-  - sends the report.text to those mail addresses
-
-
-### Frontend
-
-the architecture of the frontend is loosely oriented off [Model View
-Controller](https://blog.codinghorror.com/understanding-model-view-controller/).
-
-user.py (Model)
-- high-level interface to talk to the database
-- database calls; almost all values in the database are specific to a city/user
-- user.py is also a Class for frontend web authentication
-- user.py keeps the user-id, through which the frontend tracks authentication
-
-db.py (Model)
-- DB-Layout; creates the database if it doesn't exist yet.
-- holds some database calls which are not city-specific.
-
-frontend.py (Controller): bottle web application
-- handles POST/GET requests
-- talks to the database through user.py
-- everyone can look at the pages, and register
-- but only authenticated users can login and change settings
-
-session.py: User Authentication
-- takes care of session cookies and "403 unauthenticated" error messages
-
-sendmail.py: helper script to send mails
-- sends all mails the frontend, backend, and bots need to send
-
-static/
-- css, images, javascript for the login form etc.
-
-template/ (view)
-- base for the HTML generation, uses the bottle-template-framework
-- wrapper.tpl is the base template for every other template
-
-
-### active_bots: how to implement a new network
-
-If you want to write a new bot, e.g. a Wire-Bot, you have to take these steps:
-
-- look for a python-library which can talk to Wire
-- the city/users have to provide authentication details; this needs a form in
-  the settings
-  - depending on the network either a password, a token, or an implementation
-    of the OAuth login flow
-- the backend needs to crawl messages from the network, & post reports to the
-  network
-
-Files you need to change:
-
-1. active_bots/wire.py - crawl & post functions
-2. settings.tpl - form to authenticate to the network & possible network specific settings.
-3. frontend.py - routes for the forms you added to settings.tpl
-4. db.py - database layout, to store the account credentials/tokens, and to save which message you have last seen
-5. user.py - database calls to get or set values
-

active_bots/mailbot.py

@@ -10,7 +10,7 @@ from bot import Bot
 from config import config
 from db import db
 
-logger = logging.getLogger("main")
+logger = logging.getLogger(__name__)
 
 
 class Mailbot(Bot):
@@ -58,16 +58,30 @@ def make_report(msg, user):
     date = get_date_from_header(msg['Date'])
 
     author = msg['From']  # get mail author from email header
-    for part in msg.walk():
-        if part.get_content_type() == 'text/plain':
-            text = part.get_payload()
-        elif part.get_content_type() == 'text/html':
-            text = re.sub(r'<[^>]*>', '', msg.get_payload())
-    try:
-        post = report.Report(author, "mail", text, None, date)
-    except UnboundLocalError:
-        logger.error('No suitable message body')
-        return
+
+    if msg.is_multipart():
+        text = []
+        for part in msg.get_payload():
+            if part.get_content_type() == "text":
+                text.append(part.get_payload())
+            elif part.get_content_type() == "application/pgp-signature":
+                pass  # ignore PGP signatures
+            elif part.get_content_type() == "multipart/mixed":
+                for p in part:
+                    if isinstance(p, str):
+                        text.append(p)
+                    elif p.get_content_type() == "text":
+                        text.append(part.get_payload())
+                    else:
+                        logger.error("unknown MIMEtype: " +
+                                     p.get_content_type())
+            else:
+                logger.error("unknown MIMEtype: " +
+                             part.get_content_type())
+        text = '\n'.join(text)
+    else:
+        text = msg.get_payload()
+    post = report.Report(author, "mail", text, None, date)
     user.save_seen_mail(date)
     return post
 

active_bots/mastodonbot.py

@@ -2,12 +2,12 @@
 
 from bot import Bot
 import logging
-import mastodon
+from mastodon import Mastodon, MastodonServerError
 import re
 from report import Report
 
 
-logger = logging.getLogger("main")
+logger = logging.getLogger(__name__)
 
 
 class MastodonBot(Bot):
@@ -19,93 +19,42 @@ class MastodonBot(Bot):
         """
         mentions = []
         try:
-            m = mastodon.Mastodon(*user.get_masto_credentials())
+            m = Mastodon(*user.get_masto_credentials())
         except TypeError:
-            # No Mastodon Credentials in database.
+            # logger.error("No Mastodon Credentials in database.", exc_info=True)
             return mentions
         try:
             notifications = m.notifications()
-        except mastodon.MastodonNetworkError:
-            logger.error("Mastodon Network Error.")
-            return mentions
-        except mastodon.MastodonAPIError:
-            try:
-                logger.error("Mastodon API Error: " + m.instance()['urls']['streaming_api'] + ", city: " + str(user.uid))
-            except mastodon.MastodonServerError:
-                logger.error("Mastodon Server Error 500, can't get instance.")
-            except mastodon.MastodonVersionError:
-                logger.error("Mastodon Server Error 500, server version too low.")
-            return mentions
-        except mastodon.MastodonInternalServerError:
-            try:
-                logger.error("Mastodon Error: 500. Server: " + m.instance()['urls']['streaming_api'])
-            except mastodon.MastodonServerError:
-                logger.error("Mastodon Server Error 500, can't get instance.")
-            except mastodon.MastodonVersionError:
-                logger.error("Mastodon Server Error 500, server version too low.")
-            return mentions
-        except mastodon.MastodonBadGatewayError:
-            try:
-                logger.error("Mastodon Error: 502. Server: " + m.instance()['urls']['streaming_api'])
-            except mastodon.MastodonServerError:
-                logger.error("Mastodon Server Error 502, can't get instance.")
-            except mastodon.MastodonVersionError:
-                logger.error("Mastodon Server Error 502, server version too low.")
-            return mentions
-        except mastodon.MastodonServiceUnavailableError:
-            try:
-                logger.error("Mastodon Error: 503. Server: " + m.instance()['urls']['streaming_api'])
-            except mastodon.MastodonServerError:
-                logger.error("Mastodon Server Error 503, can't get instance.")
-            except mastodon.MastodonVersionError:
-                logger.error("Mastodon Server Error 503, server version too low.")
-            return mentions
-        except mastodon.MastodonGatewayTimeoutError:
-            try:
-                logger.error("Mastodon Error: 504. Server: " + m.instance()['urls']['streaming_api'])
-            except mastodon.MastodonServerError:
-                logger.error("Mastodon Server Error 504, can't get instance.")
-            except mastodon.MastodonVersionError:
-                logger.error("Mastodon Server Error 504, server version too low.")
-            return mentions
-        except mastodon.MastodonServerError:
-            try:
-                logger.error("Unknown Mastodon Server Error. Server: " + m.instance()['urls']['streaming_api'], exc_info=True)
-            except mastodon.MastodonServerError:
-                logger.error("Unknown Mastodon Server Error.", exc_info=True)
-            except mastodon.MastodonVersionError:
-                logger.error("Unknown Mastodon Server Error.", exc_info=True)
+        except MastodonServerError:
+            logger.error("Unknown Mastodon API Error: 502")
             return mentions
         for status in notifications:
-            try:
-                if (status['type'] == 'mention' and
+            if (status['type'] == 'mention' and
                     not user.toot_is_seen(status['status']['uri'])):
-                    # save state
-                    user.toot_witness(status['status']['uri'])
-                    # add mention to mentions
-                    text = re.sub(r'<[^>]*>', '', status['status']['content'])
-                    text = re.sub(
-                        "(?<=^|(?<=[^a-zA-Z0-9-_.]))@([A-Za-z]+[A-Za-z0-9-_]+)",
-                        "", text)
-                    if status['status']['visibility'] == 'public':
-                        mentions.append(Report(status['account']['acct'],
-                                               self,
-                                               text,
-                                               status['status']['id'],
-                                               status['status']['created_at']))
-                    else:
-                        mentions.append(Report(status['account']['acct'],
-                                               'mastodonPrivate',
-                                               text,
-                                               status['status']['id'],
-                                               status['status']['created_at']))
-            except TypeError:
-                pass
+                # save state
+                user.toot_witness(status['status']['uri'])
+                # add mention to mentions
+                text = re.sub(r'<[^>]*>', '', status['status']['content'])
+                text = re.sub(
+                    "(?<=^|(?<=[^a-zA-Z0-9-_.]))@([A-Za-z]+[A-Za-z0-9-_]+)",
+                    "", text)
+                if status['status']['visibility'] == 'public':
+                    mentions.append(Report(status['account']['acct'],
+                                           self,
+                                           text,
+                                           status['status']['id'],
+                                           status['status']['created_at']))
+                else:
+                    mentions.append(Report(status['account']['acct'],
+                                           'mastodonPrivate',
+                                           text,
+                                           status['status']['id'],
+                                           status['status']['created_at']))
         return mentions
 
     def post(self, user, report):
         try:
-            m = mastodon.Mastodon(*user.get_masto_credentials())
+            m = Mastodon(*user.get_masto_credentials())
         except TypeError:
             return  # no mastodon account for this user.
         if report.source == self:

active_bots/telegrambot.py

@@ -3,7 +3,8 @@ import logging
 from report import Report
 from twx.botapi import TelegramBot as Telegram
 
-logger = logging.getLogger("main")
+
+logger = logging.getLogger(__name__)
 
 
 class TelegramBot(Bot):
@@ -12,10 +13,9 @@ class TelegramBot(Bot):
         seen_tg = user.get_seen_tg()
         try:
             updates = tb.get_updates(offset=seen_tg + 1,
-                                     allowed_updates="message",
-                                     timeout=5).wait()
+                                     allowed_updates="message").wait()
         except TypeError:
-            updates = tb.get_updates(timeout=5).wait()
+            updates = tb.get_updates().wait()
         reports = []
         if updates == None:
             return reports
@@ -23,7 +23,6 @@ class TelegramBot(Bot):
             # return when telegram returns an error code
             if update in [303, 404, 420, 500, 502]:
                 return reports
-            # log unusual telegram error messages
             if isinstance(update, int):
                 try:
                     logger.error("City " + str(user.uid) +
@@ -32,25 +31,13 @@ class TelegramBot(Bot):
                 except TypeError:
                     logger.error("Unknown Telegram error code: " + str(update))
                 return reports
-            # save the last message, so it doesn't get crawled again
             user.save_seen_tg(update.update_id)
-            # skip if message is None
-            if update.message is None:
-                continue
-            # complain if message is a photo
-            if update.message.photo is not None:
+            if update.message.photo:
                 tb.send_message(
                     update.message.sender.id,
                     "Sending Photos is not supported for privacy reasons. Can "
                     "you describe it as text instead?")
                 continue
-            # complain if message is a media file
-            if update.message.text is None:
-                tb.send_message(
-                    update.message.sender.id,
-                    "We only support text reporting for privacy reasons. Can "
-                    "you describe it as text instead?")
-                continue
             if update.message.text.lower() == "/start":
                 user.add_telegram_subscribers(update.message.sender.id)
                 tb.send_message(

active_bots/twitterDMs.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+
+import logging
+import tweepy
+import re
+import requests
+import report
+from time import time
+from bot import Bot
+
+
+logger = logging.getLogger(__name__)
+
+
+class TwitterDMListener(Bot):
+
+    def get_api(self, user):
+        keys = user.get_twitter_credentials()
+        auth = tweepy.OAuthHandler(consumer_key=keys[0],
+                                   consumer_secret=keys[1])
+        auth.set_access_token(keys[2],  # access_token_key
+                              keys[3])  # access_token_secret
+        return tweepy.API(auth, wait_on_rate_limit=True)
+
+    def crawl(self, user):
+        """
+        crawls all Tweets which mention the bot from the twitter rest API.
+
+        :return: reports: (list of report.Report objects)
+        """
+        reports = []
+        try:
+            if user.get_last_twitter_request() + 60 > time():
+                return reports
+        except TypeError:
+            user.set_last_twitter_request(time())
+        try:
+            api = self.get_api(user)
+        except TypeError:
+            return reports  # no twitter account for this user.
+        last_dm = user.get_seen_dm()
+        try:
+            if last_dm is None:
+                mentions = api.direct_messages()
+            else:
+                mentions = api.direct_messages(since_id=last_dm[0])
+            user.set_last_twitter_request(time())
+            for status in mentions:
+                text = re.sub(
+                       "(?<=^|(?<=[^a-zA-Z0-9-_\.]))@([A-Za-z]+[A-Za-z0-9-_]+)",
+                       "", status.text)
+                reports.append(report.Report(status.author.screen_name,
+                                             "twitterDM",
+                                             text,
+                                             status.id,
+                                             status.created_at))
+            user.save_seen_dm(last_dm)
+            return reports
+        except tweepy.RateLimitError:
+            logger.error("Twitter API Error: Rate Limit Exceeded",
+                         exc_info=True)
+            # :todo implement rate limiting
+        except requests.exceptions.ConnectionError:
+            logger.error("Twitter API Error: Bad Connection", exc_info=True)
+        except tweepy.TweepError as terror:
+            # Waiting for https://github.com/tweepy/tweepy/pull/1109 to get
+            # merged, so direct messages work again
+            if terror.api_code == 34:
+                return reports
+            logger.error("Twitter API Error: General Error", exc_info=True)
+        return reports
+
+    def post(self, user, report):
+        pass

active_bots/twitterbot.py

@@ -9,7 +9,7 @@ import report
 from bot import Bot
 
 
-logger = logging.getLogger("main")
+logger = logging.getLogger(__name__)
 
 
 class TwitterBot(Bot):
@@ -67,13 +67,13 @@ class TwitterBot(Bot):
         except requests.exceptions.ConnectionError:
             logger.error("Twitter API Error: Bad Connection", exc_info=True)
         except tweepy.TweepError:
-            logger.error("Twitter API Error: General Error. User: " + str(user.uid), exc_info=True)
+            logger.error("Twitter API Error: General Error", exc_info=True)
         return []
 
     def post(self, user, report):
         try:
             api = self.get_api(user)
-        except TypeError:
+        except IndexError:
             return  # no twitter account for this user.
         try:
             if report.source == self:

backend.py

@@ -5,7 +5,7 @@ from config import config
 from db import db
 import logging
 from sendmail import sendmail
-from time import sleep
+
 
 def shutdown():
     try:
@@ -16,14 +16,11 @@ def shutdown():
 
 
 if __name__ == '__main__':
-    logger = logging.getLogger("main")
-    logger.setLevel(logging.DEBUG)
+    logger = logging.getLogger()
     fh = logging.FileHandler('/var/log/ticketfrei/backend.log')
-    formatter = logging.Formatter('%(asctime)s %(levelname)8s: %(message)s')
-    fh.setFormatter(formatter)
+    fh.setLevel(logging.DEBUG)
     logger.addHandler(fh)
 
-    logger.info("Backend Daemon was started...")
 
     bots = []
     for ActiveBot in active_bots.__dict__.values():
@@ -34,14 +31,12 @@ if __name__ == '__main__':
         while True:
             for user in db.active_users:
                 for bot in bots:
-                    sleep(1)
                     reports = bot.crawl(user)
                     for status in reports:
                         if not user.is_appropriate(status):
                             logger.info("Inaproppriate message: %d %s %s" % (user.uid, status.author, status.text))
                             continue
                         for bot2 in bots:
-                            sleep(1)
                             bot2.post(user, status)
                             logger.info("Resent: %d %s %s" % (user.uid, status.author, status.text))
     except Exception:

bots/mail/settings.tpl

@@ -0,0 +1,19 @@
+
+<div>
+    <h2>Edit your mail subscription page</h2>
+    <p>
+        There is also a page where users can subscribe to mail notifications:
+        <a href="/city/mail/{{city}}" target="_blank">Ticketfrei {{city}}</a>.
+        You can change what your users will read there, and adjust it to your
+        needs.
+    </p>
+    <p>
+        So this is the default text we suggest:
+    </p>
+    <form action="/settings/mail_md" method="post">
+        <textarea id="mail_md" rows="20" cols="70" name="mail_md" wrap="physical">{{mail_md}}</textarea>
+        <input name='csrf' value='{{csrf}}' type='hidden' />
+        <input name='confirm' value='Save' type='submit'/>
+    </form>
+</div>
+

bots/mastodon/settings.tpl

@@ -0,0 +1,51 @@
+
+<section>
+    <h2>Log in with Mastodon</h2>
+        <form action="/login/mastodon" method='post'>
+            <label for="email">E-Mail of your Mastodon-Account</label>
+            <input type="text" placeholder="Enter Email" name="email" id="email" required>
+
+            <label for="pass">Mastodon Password</label>
+            <input type="password" placeholder="Enter Password" name="pass" id="pass" required>
+
+            <label>Mastodon instance:
+                <input type='text' name='instance_url' list='instances' placeholder='social.example.net'/>
+            </label>
+            <datalist id='instances'>
+                <option value=''>
+                <option value='anticapitalist.party'>
+                <option value='awoo.space'>
+                <option value='cybre.space'>
+                <option value='mastodon.social'>
+                <option value='glitch.social'>
+                <option value='botsin.space'>
+                <option value='witches.town'>
+                <option value='social.wxcafe.net'>
+                <option value='monsterpit.net'>
+                <option value='mastodon.xyz'>
+                <option value='a.weirder.earth'>
+                <option value='chitter.xyz'>
+                <option value='sins.center'>
+                <option value='dev.glitch.social'>
+                <option value='computerfairi.es'>
+                <option value='niu.moe'>
+                <option value='icosahedron.website'>
+                <option value='hostux.social'>
+                <option value='hyenas.space'>
+                <option value='instance.business'>
+                <option value='mastodon.sdf.org'>
+                <option value='pawoo.net'>
+                <option value='pouet.it'>
+                <option value='scalie.business'>
+                <option value='sleeping.town'>
+                <option value='social.koyu.space'>
+                <option value='sunshinegardens.org'>
+                <option value='vcity.network'>
+                <option value='octodon.social'>
+                <option value='soc.ialis.me'>
+            </datalist>
+            <input name='csrf' value='{{csrf}}' type='hidden' />
+            <input name='confirm' value='Log in' type='submit'/>
+        </form>
+</section>
+

bots/telegram/settings.tpl

@@ -0,0 +1,23 @@
+
+<%
+# todo: hide this part, if there is already a telegram bot connected.
+%>
+<div>
+    <h2>Connect with Telegram</h2>
+    <p>
+        If you have a Telegram account, you can register a bot there. Just
+        write to @botfather. There are detailed instructions on
+        <a href="https://botsfortelegram.com/project/the-bot-father/" target="_blank">
+        Bots for Telegram</a>.
+    </p>
+    <p>
+        The botfather will give you an API key - with the API key, Ticketfrei
+        can use the Telegram bot. Enter it here:
+    </p>
+    <form action="/settings/telegram" method="post">
+        <input type="text" name="apikey" placeholder="Telegram bot API key" id="apikey">
+        <input name='csrf' value='{{csrf}}' type='hidden' />
+        <input name='confirm' value='Login with Telegram' type='submit'/>
+    </form>
+</div>
+

bots/twitter/settings.tpl

@@ -0,0 +1,10 @@
+
+<a class='button' style="padding: 1.5em;" href="/login/twitter">
+    <picture>
+        <source type='image/webp' sizes='20px' srcset="/static-cb/1517673283/twitter-20.webp 20w,/static-cb/1517673283/twitter-40.webp 40w,/static-cb/1517673283/twitter-80.webp 80w,"/>
+        <source type='image/png' sizes='20px' srcset="/static-cb/1517673283/twitter-20.png 20w,/static-cb/1517673283/twitter-40.png 40w,/static-cb/1517673283/twitter-80.png 80w,"/>
+        <img src="https://patriciaannbridewell.files.wordpress.com/2014/04/official-twitter-logo-tile.png" alt="" />
+    </picture>
+    Log in with Twitter
+</a>
+

db.py

@@ -1,13 +1,12 @@
 from config import config
 import jwt
 import logging
-from os import urandom, system
+from os import urandom
 from pylibscrypt import scrypt_mcf
 import sqlite3
-from time import sleep, time
 
 
-logger = logging.getLogger("main")
+logger = logging.getLogger(__name__)
 
 
 class DB(object):
@@ -20,20 +19,7 @@ class DB(object):
         return self.cur.execute(*args, **kwargs)
 
     def commit(self):
-        start_time = time()
-        while 1:
-            try:
-                self.conn.commit()
-                break
-            except sqlite3.OperationalError as error:
-                # another thread may be writing, give it a chance to finish
-                sleep(0.5)
-                logger.exception()
-                if time() - start_time > 5:
-                    # if it takes this long, something is wrong
-                    system("rcctl restart frontend_daemon")
-                    logger.warning("frontend_daemon is getting restarted")
-                    self.conn.commit()
+        self.conn.commit()
 
     def close(self):
         self.conn.close()
@@ -259,10 +245,6 @@ u\d\d?"""
             uid = json['uid']
         with open("/etc/aliases", "a+") as f:
             f.write(city + ": " + config["mail"]["mbox_user"] + "\n")
-            try:
-                os.system("newaliases")
-            except:
-                logger.exception()
         self.execute("INSERT INTO email (user_id, email) VALUES(?, ?);",
                      (uid, json['email']))
         self.execute("""INSERT INTO telegram_accounts (user_id, apikey,

deployment/backend_daemon

@@ -1,15 +0,0 @@
-#!/bin/ksh
-
-daemon="/usr/local/bin/python3 /srv/ticketfrei/backend.py"
-daemon_user="root"
-
-. /etc/rc.d/rc.subr
-
-rc_bg=YES
-rc_reload=NO
-
-rc_start() {
-        rc_exec "cd /srv/ticketfrei; /usr/bin/nice -n15 ${daemon} ${daemon_flags}"
-}
-
-rc_cmd $1

deployment/borgbackup.sh

@@ -1,15 +0,0 @@
-#!/bin/ksh
-. /etc/borg-env
-export BORG_REPO=nathan@nephilim:repositories-borg/ticketfrei
-export BORG_RSH="ssh \
--o TCPKeepAlive=no \
--o ServerAliveInterval=15 \
--o ServerAliveCountMax=10 \
--o Compression=no"
-
-rcctl stop backend_daemon
-rcctl stop frontend_daemon
-/usr/local/bin/borg create --stats ::'backup{now:%Y%m%d-%H%M}' /srv/ticketfrei /var/ticketfrei /etc
-rcctl start backend_daemon
-rcctl start frontend_daemon
-

deployment/example.org.conf

@@ -0,0 +1,31 @@
+server {
+
+    listen 443 ssl;
+    server_name example.org;
+    ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
+    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
+    ssl_ciphers         HIGH:!aNULL:!MD5;
+
+    access_log /var/log/nginx/example.org_access.log;
+    error_log /var/log/nginx/example.org_error.log;
+
+    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
+
+    location / {
+        include uwsgi_params;
+
+        uwsgi_pass unix:///var/run/ticketfrei/ticketfrei.sock;
+    }
+
+    location /.well-known/acme-challenge {
+        root /var/www/acme;
+    }
+}
+server {
+    listen 80;
+    listen [::]:80;
+    server_name example.org;
+    return 301 https://$server_name$request_uri;
+}
+

deployment/frontend_daemon

@@ -1,15 +0,0 @@
-#!/bin/ksh
-
-daemon="/usr/local/bin/python3 /srv/ticketfrei/frontend.py"
-daemon_user="frontend"
-
-. /etc/rc.d/rc.subr
-
-rc_bg=YES
-rc_reload=NO
-
-rc_start() {
-        rc_exec "env > /tmp/envars; cd /srv/ticketfrei; ${daemon} ${daemon_flags}"
-}
-
-rc_cmd $1

deployment/ticketfrei-backend.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Ticketfrei Backend
+After=syslog.target network.target
+
+[Service]
+WorkingDirectory=/srv/ticketfrei
+ExecStart=/srv/ticketfrei/bin/python3 backend.py
+# Requires systemd version 211 or newer
+#RuntimeDirectory=uwsgi
+Restart=always
+KillSignal=SIGQUIT
+Type=simple
+StandardError=syslog
+NotifyAccess=all
+
+[Install]
+WantedBy=multi-user.target

deployment/ticketfrei-web.service

@@ -0,0 +1,17 @@
+[Unit]
+Description=Ticketfrei Web Application
+After=syslog.target network.target
+
+[Service]
+WorkingDirectory=/srv/ticketfrei
+ExecStart=/usr/bin/uwsgi --ini /srv/ticketfrei/deployment/uwsgi.ini
+# Requires systemd version 211 or newer
+RuntimeDirectory=uwsgi
+Restart=always
+KillSignal=SIGQUIT
+Type=notify
+StandardError=syslog
+NotifyAccess=all
+
+[Install]
+WantedBy=multi-user.target

frontend.py

@@ -1,5 +1,6 @@
 #!/usr/bin/env python3
 import bottle
+from os import listdir, path
 from bottle import get, post, redirect, request, response, view
 from config import config
 from db import db

requirements.txt

@@ -1,9 +0,0 @@
-tweepy
-pytoml
-Mastodon.py
-bottle
-pyjwt
-pylibscrypt
-Markdown
-twx
-gitpython

sendmail.py

@@ -8,7 +8,7 @@ import smtplib
 from socket import getfqdn
 
 
-logger = logging.getLogger("main")
+logger = logging.getLogger(__name__)
 
 
 def sendmail(to, subject, city=None, body=''):

sqlalchemy.py

@@ -0,0 +1,160 @@
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy.engine import create_engine
+from sqlalchemy.schema import Table, Column, ForeignKey, UniqueConstraint
+from sqlalchemy.types import Integer, String, DateTime, BLOB, REAL
+#from sqlalchemy.orm import relationship, backref
+
+# Get Base class where table objects inherit from
+Base = declarative_base()
+engine = create_engine("sqlite:///:memory:")
+
+
+class User(Base):
+    __tablename__ = 'user'
+    id = Column(Integer, primary_key=True)
+    passhash = Column(String)
+    enabled = Column(Integer, default=1)
+
+    def __repr__(self):
+        return '<User(id=%s, passhash=%s, enabled=%s)>' % (
+            self.id, self.passhash, self.enabled)
+
+
+class Email(Base):
+    __tablename__ = 'email'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    email = Column(String)
+
+    # necessary? https://docs.sqlalchemy.org/en/13/orm/tutorial.html#building-a-relationship
+    # user = relationship(User, back_populates='email')
+
+# necessary?
+# User.email = relationship('email', order_by=Email.id, back_populates='user')
+
+class TriggerPatterns(Base):
+    __tablename__ = 'triggerpatterns'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    patterns = Column(String)
+
+class BadWords(Base):
+    __tablename__ = 'badwords'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    words = Column(String)
+
+class MastodonInstances(Base):
+    __tablename__ = 'mastodon_instances'
+    id = Column(Integer, primary_key=True)
+    instance = Column(String)
+    client_id = Column(String)
+    client_secret = Column(String)
+
+class MastodonAccounts(Base):
+    __tablename__ = 'mastodon_accounts'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    access_token = Column(String)
+    instance_id = Column(Integer, ForeignKey('mastodon_instances.id'))
+    active = Column(Integer)  # could be default=1
+
+class SeenToots(Base):
+    __tablename__ = 'seen_toots'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    toot_uri = Column(String)
+
+class SeenTelegrams(Base):
+    __tablename__ = 'seen_telegrams'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    tg_id = Column(Integer)
+
+class TwitterRequestTokens(Base):
+    __tablename__ = 'twitter_request_tokens'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    request_token = Column(String)
+    request_token_secret = Column(String)
+
+class TwitterAccounts(Base):
+    __tablename__ = 'twitter_accounts'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    client_id = Column(String)
+    client_secret = Column(String)
+    active = Column(Integer)  # could be default=1
+
+class TelegramAccounts(Base):
+    __tablename__ = 'telegram_accounts'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    apikey = Column(String)
+    active = Column(Integer)  # could be default=1
+
+class SeenTweets(Base):
+    __tablename__ = 'seen_tweets'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    tweet_id = Column(String)
+
+class SeenDMs(Base):
+    __tablename__ = 'seen_dms'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    twitter_accounts = Column(Integer, ForeignKey('twitter_accounts.id'))
+    message_id = Column(String)
+
+class TelegramSubscribers(Base):
+    __tablename__ = 'telegram_subscribers'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    subscriber_id = Column(Integer)
+
+    # how to get this to work?
+    # https://docs.sqlalchemy.org/en/13/dialects/sqlite.html#on-conflict-support-for-constraints
+    # UniqueConstraint('id', 'subscriber_id', sqlite_on_conflict='IGNORE')
+
+class Mailinglist(Base):
+    __tablename__ = 'mailinglist'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    email = Column(String)
+
+    # It would be good to have a Unique on conflict ignore here, just as in
+    # telegram_subscribers.
+
+class SeenMail(Base):
+    __tablename__ = 'seen_mail'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    mail_date = Column(REAL) # could be Datetime, too
+
+class TwitterLastRequest(Base):
+    __tablename__ = 'twitter_last_request'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    date = Column(Integer)
+
+class Cities(Base):
+    __tablename__ = 'cities'
+    id = Column(Integer, primary_key=True)
+    user_id = Column(Integer, ForeignKey('user.id'))
+    city = Column(String)
+    markdown = Column(String)
+    mail_md = Column(String)
+    masto_link = Column(String)
+    twit_link = Column(String)
+
+    # how to get this to work?
+    # https://docs.sqlalchemy.org/en/13/dialects/sqlite.html#on-conflict-support-for-constraints
+    # UniqueConstraint('id', 'city', sqlite_on_conflict='IGNORE')
+
+class Secret(Base):
+    __tablename__ = 'secret'
+    id = Column(Integer, primary_key=True)
+    secret = Column(BLOB)
+
+Base.metadata.create_all(engine)
+

template/settings.tpl

@@ -7,86 +7,15 @@
     <div id="enablebutton" style="float: right; padding: 2em;" color="red">Enable</div>
 % end
 
-<a class='button' style="padding: 1.5em;" href="/login/twitter">
-    <picture>
-        <source type='image/webp' sizes='20px' srcset="/static-cb/1517673283/twitter-20.webp 20w,/static-cb/1517673283/twitter-40.webp 40w,/static-cb/1517673283/twitter-80.webp 80w,"/>
-        <source type='image/png' sizes='20px' srcset="/static-cb/1517673283/twitter-20.png 20w,/static-cb/1517673283/twitter-40.png 40w,/static-cb/1517673283/twitter-80.png 80w,"/>
-        <img src="https://patriciaannbridewell.files.wordpress.com/2014/04/official-twitter-logo-tile.png" alt="" />
-    </picture>
-    Log in with Twitter
-</a>
-
-<section>
-    <h2>Log in with Mastodon</h2>
-        <form action="/login/mastodon" method='post'>
-            <label for="email">E-Mail of your Mastodon-Account</label>
-            <input type="text" placeholder="Enter Email" name="email" id="email" required>
-
-            <label for="pass">Mastodon Password</label>
-            <input type="password" placeholder="Enter Password" name="pass" id="pass" required>
-
-            <label>Mastodon instance:
-                <input type='text' name='instance_url' list='instances' placeholder='social.example.net'/>
-            </label>
-            <datalist id='instances'>
-                <option value=''>
-                <option value='anticapitalist.party'>
-                <option value='awoo.space'>
-                <option value='cybre.space'>
-                <option value='mastodon.social'>
-                <option value='glitch.social'>
-                <option value='botsin.space'>
-                <option value='witches.town'>
-                <option value='social.wxcafe.net'>
-                <option value='monsterpit.net'>
-                <option value='mastodon.xyz'>
-                <option value='a.weirder.earth'>
-                <option value='chitter.xyz'>
-                <option value='sins.center'>
-                <option value='dev.glitch.social'>
-                <option value='computerfairi.es'>
-                <option value='niu.moe'>
-                <option value='icosahedron.website'>
-                <option value='hostux.social'>
-                <option value='hyenas.space'>
-                <option value='instance.business'>
-                <option value='mastodon.sdf.org'>
-                <option value='pawoo.net'>
-                <option value='pouet.it'>
-                <option value='scalie.business'>
-                <option value='sleeping.town'>
-                <option value='social.koyu.space'>
-                <option value='sunshinegardens.org'>
-                <option value='vcity.network'>
-                <option value='octodon.social'>
-                <option value='soc.ialis.me'>
-            </datalist>
-            <input name='csrf' value='{{csrf}}' type='hidden' />
-            <input name='confirm' value='Log in' type='submit'/>
-        </form>
-</section>
-
 <%
-# todo: hide this part, if there is already a telegram bot connected.
+# import all the settings templates from bots/*/settings.tpl
+import os
+bots = os.listdir('bots')
+
+for bot in bots:
+    include('bots/' + bot + '/settings.tpl', csrf=csrf, city=city)
+end
 %>
-<div>
-    <h2>Connect with Telegram</h2>
-    <p>
-        If you have a Telegram account, you can register a bot there. Just
-        write to @botfather. There are detailed instructions on
-        <a href="https://botsfortelegram.com/project/the-bot-father/" target="_blank">
-        Bots for Telegram</a>.
-    </p>
-    <p>
-        The botfather will give you an API key - with the API key, Ticketfrei
-        can use the Telegram bot. Enter it here:
-    </p>
-    <form action="/settings/telegram" method="post">
-        <input type="text" name="apikey" placeholder="Telegram bot API key" id="apikey">
-        <input name='csrf' value='{{csrf}}' type='hidden' />
-        <input name='confirm' value='Login with Telegram' type='submit'/>
-    </form>
-</div>
 
 <div>
     <h2>Edit your city page</h2>
@@ -113,24 +42,6 @@
     </form>
 </div>
 
-<div>
-    <h2>Edit your mail subscription page</h2>
-    <p>
-        There is also a page where users can subscribe to mail notifications:
-        <a href="/city/mail/{{city}}" target="_blank">Ticketfrei {{city}}</a>.
-        You can change what your users will read there, and adjust it to your
-        needs.
-    </p>
-    <p>
-        So this is the default text we suggest:
-    </p>
-    <form action="/settings/mail_md" method="post">
-        <textarea id="mail_md" rows="20" cols="70" name="mail_md" wrap="physical">{{mail_md}}</textarea>
-        <input name='csrf' value='{{csrf}}' type='hidden' />
-        <input name='confirm' value='Save' type='submit'/>
-    </form>
-</div>
-
 <div>
     <h2>Edit your trigger patterns</h2>
     <p>

user.py

@@ -5,9 +5,7 @@ import jwt
 from mastodon import Mastodon
 from pylibscrypt import scrypt_mcf, scrypt_mcf_check
 from os import urandom
-import logging
 
-logger = logging.getLogger("main")
 
 class User(object):
     def __init__(self, uid):
@@ -76,7 +74,6 @@ class User(object):
                 break
         else:
             # no pattern matched
-            logger.error("Message didn't trigger goodlist: " + report.text)
             return False
         default_badwords = """
 bitch
@@ -96,15 +93,12 @@ schlitz
         db.execute("SELECT words FROM badwords WHERE user_id=?;",
                    (self.uid, ))
         badwords = db.cur.fetchone()
-        for word in report.text.lower().split():
-            if word in badwords[0].splitlines():
-                logger.error("Word " + word + " triggered the spam filter on message: " + report.text)
+        for word in report.text.lower().splitlines():
+            if word in badwords:
                 return False
-        for word in report.text.lower().split():
-            if word in default_badwords.splitlines():
-                logger.error("Word " + word + " triggered the spam filter on message: " + report.text)
+        for word in default_badwords.splitlines():
+            if word in badwords:
                 return False
-        logger.info("Valid report: " + report.text + " | username: " + report.author)
         return True
 
     def get_last_twitter_request(self):
@@ -118,16 +112,16 @@ schlitz
         db.commit()
 
     def get_telegram_credentials(self):
-        db.execute("""SELECT apikey
-                          FROM telegram_accounts
+        db.execute("""SELECT apikey 
+                          FROM telegram_accounts 
                           WHERE user_id = ? AND active = 1;""",
                    (self.uid,))
         row = db.cur.fetchone()
         return row[0]
 
     def get_telegram_subscribers(self):
-        db.execute("""SELECT subscriber_id
-                          FROM telegram_subscribers
+        db.execute("""SELECT subscriber_id 
+                          FROM telegram_subscribers 
                           WHERE user_id = ?;""",
                    (self.uid,))
         rows = db.cur.fetchall()
@@ -140,21 +134,21 @@ schlitz
         db.commit()
 
     def remove_telegram_subscribers(self, subscriber_id):
-        db.execute("""DELETE
-                          FROM telegram_subscribers
+        db.execute("""DELETE 
+                          FROM telegram_subscribers 
                           WHERE user_id = ?
                           AND subscriber_id = ?;""",
                    (self.uid, subscriber_id))
         db.commit()
 
     def get_masto_credentials(self):
-        db.execute("""SELECT access_token, instance_id
-                          FROM mastodon_accounts
+        db.execute("""SELECT access_token, instance_id 
+                          FROM mastodon_accounts 
                           WHERE user_id = ? AND active = 1;""",
                    (self.uid,))
         row = db.cur.fetchone()
-        db.execute("""SELECT instance, client_id, client_secret
-                          FROM mastodon_instances
+        db.execute("""SELECT instance, client_id, client_secret 
+                          FROM mastodon_instances 
                           WHERE id = ?;""",
                    (row[1],))
         instance = db.cur.fetchone()
@@ -272,11 +266,11 @@ schlitz
         db.commit()
 
     def get_request_token(self):
-        db.execute("""SELECT request_token, request_token_secret
-                          FROM twitter_request_tokens
+        db.execute("""SELECT request_token, request_token_secret 
+                          FROM twitter_request_tokens 
                           WHERE user_id = ?;""", (self.uid,))
         request_token = db.cur.fetchone()
-        db.execute("""DELETE FROM twitter_request_tokens
+        db.execute("""DELETE FROM twitter_request_tokens 
                           WHERE user_id = ?;""", (self.uid,))
         db.commit()
         return {"oauth_token": request_token[0],
@@ -287,8 +281,6 @@ schlitz
                            user_id, client_id, client_secret
                            ) VALUES(?, ?, ?);""",
                    (self.uid, access_token, access_token_secret))
-        db.execute("""INSERT INTO seen_tweets(user_id, tweet_id) VALUES (?, ?);""",
-                   (self.uid, 0))
         db.commit()
 
     def get_twitter_token(self):
@@ -309,8 +301,8 @@ schlitz
         db.commit()
 
     def get_mastodon_app_keys(self, instance):
-        db.execute("""SELECT client_id, client_secret
-                          FROM mastodon_instances
+        db.execute("""SELECT client_id, client_secret 
+                          FROM mastodon_instances 
                           WHERE instance = ?;""", (instance,))
         try:
             row = db.cur.fetchone()
@@ -329,8 +321,8 @@ schlitz
             return client_id, client_secret
 
     def save_masto_token(self, access_token, instance):
-        db.execute("""SELECT id
-                          FROM mastodon_instances
+        db.execute("""SELECT id 
+                          FROM mastodon_instances 
                           WHERE instance = ?;""", (instance,))
         instance_id = db.cur.fetchone()[0]
         db.execute("INSERT INTO mastodon_accounts(user_id, access_token, instance_id, active) "
@@ -366,20 +358,20 @@ Schau einfach auf das Profil unseres Bots: """ + twit_link + """
 
 Hat jemand vor kurzem etwas über Kontrolleur\*innen gepostet?
 
-* Wenn ja, dann kauf dir vllt lieber ein Ticket. In Nürnberg
+* Wenn ja, dann kauf dir vllt lieber ein Ticket. In Nürnberg 
   haben wir die Erfahrung gemacht, dass Kontis normalerweile
-  ungefähr ne Woche aktiv sind, ein paar Stunden am Tag. Wenn es
-  also in den letzten Stunden einen Bericht gab, pass lieber
+  ungefähr ne Woche aktiv sind, ein paar Stunden am Tag. Wenn es 
+  also in den letzten Stunden einen Bericht gab, pass lieber 
   auf.
 * Wenn nicht, ist es wahrscheinlich kein Problem :)
 
-Wir können natürlich nicht garantieren, dass es sicher ist,
+Wir können natürlich nicht garantieren, dass es sicher ist, 
 also pass trotzdem auf, wer auf dem Bahnsteig steht.
-Aber je mehr Leute mitmachen, desto eher kannst du dir sicher
+Aber je mehr Leute mitmachen, desto eher kannst du dir sicher 
 sein, dass wir sie finden, bevor sie uns finden.
 
 Wenn du immer direkt gewarnt werden willst, kannst du auch die
-Benachrichtigungen über E-Mail, Telegram, oder den Mastodon RSS
+Benachrichtigungen über E-Mail, Telegram, oder den Mastodon RSS 
 feed aktivieren. Entweder:
 * Gibt hier [deine E-Mail-Adresse an](/city/mail/""" + city + """)
 * Subscribe dem Telegram-Bot [@ticketfrei_""" + city + \
@@ -387,7 +379,7 @@ feed aktivieren. Entweder:
 * oder subscribe dem RSS feed von [""" + city + """](""" + masto_link + \
 """.atom?replies=false&boosts=true)
 
-Also, wenn du weniger Glück hast, und der erste bist, der einen
+Also, wenn du weniger Glück hast, und der erste bist, der einen 
 Kontrolleur sieht:
 
 ## Was mache ich, wenn ich Kontis sehe?
@@ -398,10 +390,10 @@ Ganz einfach, du schreibst es den anderen. Das geht entweder
 * über Twitter: [Link zu unserem Profil](""" + twit_link + """)
 * über Telegram an [@ticketfrei_""" + city + "_bot](https://t.me/ticketfrei_" \
                    + city + """_bot)
-* Oder per Mail an [""" + mailinglist + "](mailto:" + mailinglist + """), wenn
+* Oder per Mail an [""" + mailinglist + "](mailto:" + mailinglist + """), wenn 
   ihr kein Social Media benutzen wollt.
 
-Schreibe einfach einen Toot oder einen Tweet, der den Bot
+Schreibe einfach einen Toot oder einen Tweet, der den Bot 
 mentioned, und gib an
 
 * Wo du die Kontis gesehen hast
@@ -413,11 +405,11 @@ Zum Beispiel so:
 
 ![A toot ready to be shared](https://github.com/b3yond/ticketfrei/raw/stable1/guides/toot_screenshot.png)
 
-Der Bot wird die Nachricht dann weiterverbreiten, auch zu den
+Der Bot wird die Nachricht dann weiterverbreiten, auch zu den 
 anderen Netzwerken.
 Dann können andere Leute das lesen und sicher vor Kontis sein.
 
-Danke, dass du mithilfst, öffentlichen Verkehr für alle
+Danke, dass du mithilfst, öffentlichen Verkehr für alle 
 sicherzustellen!
 
 ## Kann ich darauf vertrauen, was random stranger from the Internet mir da erzählen?
@@ -426,31 +418,31 @@ Aber natürlich! Wir haben Katzenbilder!
 
 ![Katzenbilder!](https://lorempixel.com/550/300/cats)
 
-Glaubt besser nicht, wenn jemand postet, dass die Luft da und
+Glaubt besser nicht, wenn jemand postet, dass die Luft da und 
 da gerade rein ist.
-Das ist vielleicht sogar gut gemeint - aber klar könnte die
+Das ist vielleicht sogar gut gemeint - aber klar könnte die 
 VAG sich hinsetzen und einfach lauter Falschmeldungen posten.
 
-Aber Falschmeldungen darüber, dass gerade Kontis i-wo unterwegs
+Aber Falschmeldungen darüber, dass gerade Kontis i-wo unterwegs 
 sind?
-Das macht keinen Sinn.
-Im schlimmsten Fall kauft jmd mal eine Fahrkarte mehr - aber
+Das macht keinen Sinn. 
+Im schlimmsten Fall kauft jmd mal eine Fahrkarte mehr - aber 
 kann sonst immer schwarz fahren.
 
-Also ja - es macht Sinn, uns zu vertrauen, wenn wir sagen, wo
+Also ja - es macht Sinn, uns zu vertrauen, wenn wir sagen, wo 
 gerade Kontis sind.
 
 ## Was ist Mastodon und warum sollte ich es benutzen?
 
-Mastodon ist ein dezentrales soziales Netzwerk - so wie
+Mastodon ist ein dezentrales soziales Netzwerk - so wie 
 Twitter, nur ohne Monopol und Zentralismus.
-Ihr könnt Kurznachrichten (Toots) über alles mögliche
+Ihr könnt Kurznachrichten (Toots) über alles mögliche 
 schreiben, und euch mit anderen austauschen.
 
-Mastodon ist Open Source, Privatsphäre-freundlich und relativ
+Mastodon ist Open Source, Privatsphäre-freundlich und relativ 
 sicher vor Zensur.
 
-Um Mastodon zu benutzen, besucht diese Seite:
+Um Mastodon zu benutzen, besucht diese Seite: 
 [https://joinmastodon.org/](https://joinmastodon.org/)
         """
         mail_md = """# Immer up-to-date
@@ -460,15 +452,15 @@ zu schauen? Kein Problem. Unsere Mail Notifications benachrichtigen dich, wenn
 irgendwo Kontis gesehen werden.
 
 Wenn du uns deine E-Mail-Adresse gibst, kriegst du bei jedem Konti-Report eine
-Mail. Wenn du eine Mail-App auf dem Handy hast, so wie
+Mail. Wenn du eine Mail-App auf dem Handy hast, so wie 
 [K9Mail](https://k9mail.github.io/), kriegst du sogar eine Push Notification. So
 bist du immer Up-to-date über alles, was im Verkehrsnetz passiert.
 
 ## Keine Sorge
 
-Wir benutzen deine E-Mail-Adresse selbstverständlich für nichts anderes. Du
+Wir benutzen deine E-Mail-Adresse selbstverständlich für nichts anderes. Du 
 kannst die Benachrichtigungen jederzeit deaktivieren, mit jeder Mail wird ein
-unsubscribe-link mitgeschickt.
+unsubscribe-link mitgeschickt. 
         """
         db.execute("""INSERT INTO cities(user_id, city, markdown, mail_md,
                         masto_link, twit_link) VALUES(?,?,?,?,?,?)""",