fixing import error

don't log Mastodon 502 errors.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,27 @@
+---
+name: Bug report
+about: Create a report to help us improve
+
+---
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Actual Behavior**
+A clear and concise description of what happens.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Ticketfrei Version**
+See the commit on which Ticketfrei is running at example.org/version.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/ISSUE_TEMPLATE/something-else.md

@@ -0,0 +1,7 @@
+---
+name: Something else
+about: Other ideas?
+
+---
+
+*If your suggestion is neither a bug report nor a feature request, this is the right place. Just describe what you have in mind.*

LICENSE

@@ -1,6 +1,6 @@
 Copyright (c) 2017 Thomas L <tom@dl6tom.de>
 Copyright (c) 2017 b3yond <b3yond@riseup.net>
-Copyright (c) 2018 sid
+Copyright (c) 2018 sid <sid-sid@riseup.net>
 
 Permission to use, copy, modify, and distribute this software for any
 purpose with or without fee is hereby granted, provided that the above

README.md

@@ -3,6 +3,23 @@
 Ticketfrei is a mastodon/twitter/mail bot to dodge ticket controllers in public
 transport systems.
 
+## Mission
+
+Public transportation is meant to provide an easy and time-saving way to move
+within a region while being affordable for everybody. Unfortunately, this is
+not yet the case. Ticketfrei's approach is to **enable people to reclaim public
+transportation.**
+
+On short term we want to do this by helping users to avoid controllers and
+fines - on long term by **pressuring public transportation companies to offer
+their services free of charge**, financed by the public.
+
+Because with Ticketfrei you're able to use trains and subways for free anyway.
+Take part and create a new understanding of what public transportation could
+look like!
+
+## How It Works
+
 The functionality is simple: It retweets every tweet where it is mentioned.
 
 This leads to a community which evolves around it. If you see ticket
@@ -91,7 +108,7 @@ virtualenv -p python3 .
 Install the dependencies:
 
 ```shell
-pip install tweepy pytoml Mastodon.py bottle pyjwt pylibscrypt Markdown twx
+pip install tweepy pytoml Mastodon.py bottle pyjwt pylibscrypt Markdown twx gitpython
 ```
 
 Configure the bot:

active_bots/mailbot.py

@@ -19,7 +19,11 @@ class Mailbot(Bot):
     def crawl(self, user):
         reports = []
         # todo: adjust to actual mailbox
-        mails = mailbox.mbox("/var/mail/" + config['mail']['mbox_user'])
+        try:
+            mails = mailbox.mbox("/var/mail/" + config['mail']['mbox_user'])
+        except FileNotFoundError:
+            logger.error("No mbox file found.")
+            return reports
         for msg in mails:
             if get_date_from_header(msg['Date']) > user.get_seen_mail():
                 if user.get_city().lower() in msg['To'].lower():

active_bots/mastodonbot.py

@@ -2,7 +2,7 @@
 
 from bot import Bot
 import logging
-from mastodon import Mastodon
+import mastodon
 import re
 from report import Report
 
@@ -19,14 +19,14 @@ class MastodonBot(Bot):
         """
         mentions = []
         try:
-            m = Mastodon(*user.get_masto_credentials())
+            m = mastodon.Mastodon(*user.get_masto_credentials())
         except TypeError:
             # logger.error("No Mastodon Credentials in database.", exc_info=True)
             return mentions
         try:
             notifications = m.notifications()
-        except Exception:
-            logger.error("Unknown Mastodon API Error.", exc_info=True)
+        except mastodon.MastodonServerError:
+            logger.error("Unknown Mastodon API Error: 502")
             return mentions
         for status in notifications:
             if (status['type'] == 'mention' and
@@ -54,7 +54,7 @@ class MastodonBot(Bot):
 
     def post(self, user, report):
         try:
-            m = Mastodon(*user.get_masto_credentials())
+            m = mastodon.Mastodon(*user.get_masto_credentials())
         except TypeError:
             return  # no mastodon account for this user.
         if report.source == self:

active_bots/telegrambot.py

@@ -21,12 +21,23 @@ class TelegramBot(Bot):
             return reports
         for update in updates:
             # return when telegram returns an error code
-            if update in [303, 404, 420, 500]:
+            if update in [303, 404, 420, 500, 502]:
                 return reports
-            elif isinstance(update, int):
-                logger.error("Unknown Telegram error code: " + str(update))
+            if isinstance(update, int):
+                try:
+                    logger.error("City " + str(user.uid) +
+                                 ": Unknown Telegram error code: " +
+                                 str(update) + " - " + str(updates[1]))
+                except TypeError:
+                    logger.error("Unknown Telegram error code: " + str(update))
                 return reports
             user.save_seen_tg(update.update_id)
+            if update.message.photo:
+                tb.send_message(
+                    update.message.sender.id,
+                    "Sending Photos is not supported for privacy reasons. Can "
+                    "you describe it as text instead?")
+                continue
             if update.message.text.lower() == "/start":
                 user.add_telegram_subscribers(update.message.sender.id)
                 tb.send_message(
@@ -42,19 +53,24 @@ class TelegramBot(Bot):
             elif update.message.text.lower() == "/help":
                 tb.send_message(
                     update.message.sender.id,
-                    "Send reports here to share them with other users. Use /start and /stop to get reports or not.")
+                    "Send reports here to share them with other users. "
+                    "Use /start and /stop to get reports or not.")
                 # TODO: /help message should be set in frontend
             else:
-                reports.append(Report(update.message.sender.username, self,
-                                      update.message.text, None,
-                                      update.message.date))
+                # set report.author to "" to avoid mailbot crash
+                sender_name = update.message.sender.username
+                if sender_name is None:
+                    sender_name = ""
+
+                reports.append(Report(sender_name, self, update.message.text,
+                                      None, update.message.date))
         return reports
 
     def post(self, user, report):
         tb = Telegram(user.get_telegram_credentials())
         text = report.text
         if len(text) > 4096:
-            text = text[:4096 - 4] + u' ...'
+            text = text[:4096 - 2] + " \N{Horizontal ellipsis}"
         try:
             for subscriber_id in user.get_telegram_subscribers():
                 tb.send_message(subscriber_id, text).wait()

active_bots/twitterbot.py

@@ -73,7 +73,7 @@ class TwitterBot(Bot):
     def post(self, user, report):
         try:
             api = self.get_api(user)
-        except IndexError:
+        except TypeError:
             return  # no twitter account for this user.
         try:
             if report.source == self:

config.py

@@ -1,5 +1,70 @@
 import pytoml as toml
+import os
+
+
+def load_env():
+    """
+    load environment variables from the environment. If empty, use default
+    values from config.toml.example.
+
+    :return: config dictionary of dictionaries.
+    """
+    with open('config.toml.example') as defaultconf:
+        configdict = toml.load(defaultconf)
+
+    try:
+        if os.environ['CONSUMER_KEY'] != "":
+            configdict['twitter']['consumer_key'] = os.environ['CONSUMER_KEY']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['CONSUMER_SECRET'] != "":
+            configdict['twitter']['consumer_secret'] = os.environ['CONSUMER_SECRET']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['HOST'] != "":
+            configdict['web']['host'] = os.environ['HOST']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['PORT'] != "":
+            configdict['web']['port'] = os.environ['PORT']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['CONTACT'] != "":
+            configdict['web']['contact'] = os.environ['CONTACT']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['MBOX_USER'] != "":
+            configdict['mail']['mbox_user'] = os.environ['MBOX_USER']
+    except KeyError:
+        pass
+
+    try:
+        if os.environ['DB_PATH'] != "":
+            configdict['database']['db_path'] = os.environ['DB_PATH']
+    except KeyError:
+        pass
+
+    return configdict
+
 
 # read config in TOML format (https://github.com/toml-lang/toml#toml)
-with open('config.toml') as configfile:
-    config = toml.load(configfile)
+try:
+    with open('config.toml') as configfile:
+        config = toml.load(configfile)
+except FileNotFoundError:
+    config = load_env()
+
+if __name__ == "__main__":
+    for category in config:
+        for key in config[category]:
+            print(key + "=" + str(config[category][key]))

config.toml.example

@@ -10,10 +10,7 @@ port = 80
 contact = "b3yond@riseup.net"
 
 [mail]
-mailserver = "smtp.riseup.net"
-user = "user"
-passphrase = "sup3rs3cur3"
-mbox = "root"
+mbox_user = "root"
 
 [database]
 db_path = "/var/ticketfrei/db.sqlite"

db.py

@@ -14,7 +14,6 @@ class DB(object):
         self.conn = sqlite3.connect(dbfile)
         self.cur = self.conn.cursor()
         self.create()
-        self.secret = self.get_secret()
 
     def execute(self, *args, **kwargs):
         return self.cur.execute(*args, **kwargs)
@@ -115,13 +114,6 @@ class DB(object):
                 FOREIGN KEY(twitter_accounts_id)
                     REFERENCES twitter_accounts(id)
             );
-            CREATE TABLE IF NOT EXISTS telegram_accounts (
-                id          INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
-                user_id     INTEGER,
-                api_token   TEXT,
-                active      INTEGER,
-                FOREIGN KEY(user_id) REFERENCES user(id)
-            );
             CREATE TABLE IF NOT EXISTS telegram_subscribers (
                 id          INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
                 user_id     INTEGER,
@@ -196,7 +188,7 @@ class DB(object):
             'passhash': scrypt_mcf(
                 password.encode('utf-8')
             ).decode('ascii')
-        }, self.secret).decode('ascii')
+        }, self.get_secret()).decode('ascii')
 
     def mail_subscription_token(self, email, city):
         """
@@ -210,17 +202,17 @@ class DB(object):
         token = jwt.encode({
             'email': email,
             'city': city
-        }, self.secret).decode('ascii')
+        }, self.get_secret()).decode('ascii')
         return token
 
     def confirm_subscription(self, token):
-        json = jwt.decode(token, self.secret)
+        json = jwt.decode(token, self.get_secret())
         return json['email'], json['city']
 
     def confirm(self, token, city):
         from user import User
         try:
-            json = jwt.decode(token, self.secret)
+            json = jwt.decode(token, self.get_secret())
         except jwt.DecodeError:
             return None  # invalid token
         if 'passhash' in json.keys():

frontend.py

@@ -56,11 +56,22 @@ def register_post():
 @get('/confirm/<city>/<token>')
 @view('template/propaganda.tpl')
 def confirm(city, token):
+    # check whether city already exists
+    if db.by_city(city):
+        return dict(error='This Account was already confirmed, please try '
+                          'signing in.')
     # create db-entry
     if db.confirm(token, city):
         # :todo show info "Account creation successful."
         redirect('/settings')
-    return dict(error='Email confirmation failed.')
+    return dict(error='Account creation failed. Please try to register again.')
+
+
+@get('/version')
+def version():
+    import git
+    repo = git.Repo(search_parent_directories=True)
+    return repo.head.object.hexsha
 
 
 @post('/login')
@@ -167,9 +178,10 @@ def register_telegram(user):
     return city_page(user.get_city(), info="Thanks for registering Telegram!")
 
 
-@get('/api/state')
-def api_enable(user):
-    return user.state()
+# unused afaik
+#@get('/api/state')
+#def api_enable(user):
+#    return user.state()
 
 
 @get('/static/<filename:path>')
@@ -186,6 +198,7 @@ def guides(filename):
 def logout():
     # clear auth cookie
     response.set_cookie('uid', '', expires=0, path="/")
+    response.set_cookie('csrf', '', expires=0, path="/")
     # :todo show info "Logout successful."
     redirect('/')
 
@@ -244,11 +257,6 @@ def login_mastodon(user):
     try:
         access_token = m.log_in(masto_email, masto_pass)
         user.save_masto_token(access_token, instance_url)
-
-        # Trying to set the seen_toot to 0, thereby initializing it.
-        # It should work now, but has default values. Not sure if I need them.
-        user.init_seen_toot(instance_url)
-
         return city_page(user.get_city(), info='Thanks for supporting decentralized social networks!')
     except Exception:
         logger.error('Login to Mastodon failed.', exc_info=True)
@@ -264,7 +272,6 @@ application = bottle.default_app()
 bottle.install(SessionPlugin('/'))
 
 if __name__ == '__main__':
-    # testing only
-    bottle.run(host=config["web"]["host"], port=config["web"]["port"])
+    bottle.run(host="0.0.0.0", port=config["web"]["port"])
 else:
     application.catchall = False

sendmail.py

@@ -27,5 +27,5 @@ def sendmail(to, subject, city=None, body=''):
 
 # For testing:
 if __name__ == '__main__':
-    sendmail(config['mail']['contact'], "Test Mail",
+    sendmail(config['web']['contact'], "Test Mail",
              body="This is a test mail.")

session.py

@@ -1,4 +1,4 @@
-from bottle import redirect, request
+from bottle import redirect, request, abort, response
 from db import db
 from functools import wraps
 from inspect import Signature
@@ -17,10 +17,14 @@ class SessionPlugin(object):
         if self.keyword in Signature.from_callable(route.callback).parameters:
             @wraps(callback)
             def wrapper(*args, **kwargs):
-                uid = request.get_cookie('uid', secret=db.secret)
+                uid = request.get_cookie('uid', secret=db.get_secret())
                 if uid is None:
                     return redirect(self.loginpage)
                 kwargs[self.keyword] = User(uid)
+                if request.method == 'POST':
+                    if request.forms['csrf'] != request.get_cookie('csrf',
+                                                        secret=db.get_secret()):
+                        abort(400)
                 return callback(*args, **kwargs)
 
             return wrapper

static/css/style.css

@@ -1,6 +1,6 @@
 body {
   background-image: url(/static/img/ticketfrei-og-image.jpg);
-  background-size: 50%;
+  background-height: 100%;
   font-family: Verdana, Arial, Helvetica, sans-serif;
   font-size: 12pt;
   line-height: 1.5em;

template/settings.tpl

@@ -61,6 +61,7 @@
                 <option value='octodon.social'>
                 <option value='soc.ialis.me'>
             </datalist>
+            <input name='csrf' value='{{csrf}}' type='hidden' />
             <input name='confirm' value='Log in' type='submit'/>
         </form>
 </section>
@@ -82,6 +83,7 @@
     </p>
     <form action="/settings/telegram" method="post">
         <input type="text" name="apikey" placeholder="Telegram bot API key" id="apikey">
+        <input name='csrf' value='{{csrf}}' type='hidden' />
         <input name='confirm' value='Login with Telegram' type='submit'/>
     </form>
 </div>
@@ -106,6 +108,7 @@
     </p>
     <form action="/settings/markdown" method="post">
         <textarea id="markdown" rows="20" cols="70" name="markdown" wrap="physical">{{markdown}}</textarea>
+        <input name='csrf' value='{{csrf}}' type='hidden' />
         <input name='confirm' value='Save' type='submit'/>
     </form>
 </div>
@@ -123,6 +126,7 @@
     </p>
     <form action="/settings/mail_md" method="post">
         <textarea id="mail_md" rows="20" cols="70" name="mail_md" wrap="physical">{{mail_md}}</textarea>
+        <input name='csrf' value='{{csrf}}' type='hidden' />
         <input name='confirm' value='Save' type='submit'/>
     </form>
 </div>
@@ -137,6 +141,7 @@
     </p>
     <form action="/settings/goodlist" method="post">
         <textarea id="goodlist" rows="8" cols="70" name="goodlist" wrap="physical">{{triggerwords}}</textarea>
+        <input name='csrf' value='{{csrf}}' type='hidden' />
         <input name='confirm' value='Submit' type='submit'/>
     </form>
 </div>
@@ -151,6 +156,7 @@
     </p>
     <form action="/settings/blocklist" method="post">
         <textarea id="blocklist" rows="8" cols="70" name="blocklist" wrap="physical">{{badwords}}</textarea>
+        <input name='csrf' value='{{csrf}}' type='hidden' />
         <input name='confirm' value='Submit' type='submit'/>
     </form>
 </div>

user.py

@@ -1,16 +1,24 @@
 from config import config
-from bottle import response
+from bottle import response, request
 from db import db
 import jwt
 from mastodon import Mastodon
 from pylibscrypt import scrypt_mcf, scrypt_mcf_check
+from os import urandom
 
 
 class User(object):
     def __init__(self, uid):
         # set cookie
-        response.set_cookie('uid', uid, secret=db.secret, path='/')
+        response.set_cookie('uid', uid, secret=db.get_secret(), path='/')
         self.uid = uid
+        response.set_cookie('csrf', self.get_csrf(), db.get_secret(), path='/')
+
+    def get_csrf(self):
+        csrf_token = request.get_cookie('csrf', secret=db.get_secret())
+        if not csrf_token:
+            csrf_token = str(urandom(32))
+        return csrf_token
 
     def check_password(self, password):
         db.execute("SELECT passhash FROM user WHERE id=?;", (self.uid,))
@@ -55,7 +63,7 @@ class User(object):
         return jwt.encode({
             'email': email,
             'uid': self.uid
-        }, db.secret).decode('ascii')
+        }, db.get_secret()).decode('ascii')
 
     def is_appropriate(self, report):
         db.execute("SELECT patterns FROM triggerpatterns WHERE user_id=?;",
@@ -235,6 +243,7 @@ schlitz
         # - mail_md
         # - goodlist
         # - blocklist
+        # - csrf
         # - logged in with twitter?
         # - logged in with mastodon?
         # - enabled?
@@ -244,7 +253,8 @@ schlitz
                     mail_md=citydict['mail_md'],
                     triggerwords=self.get_trigger_words(),
                     badwords=self.get_badwords(),
-                    enabled=self.enabled)
+                    enabled=self.enabled,
+                    csrf=self.get_csrf())
 
     def save_request_token(self, token):
         db.execute("""INSERT INTO
@@ -300,7 +310,7 @@ schlitz
             client_secret = row[1]
             return client_id, client_secret
         except TypeError:
-            app_name = "ticketfrei" + str(db.secret)[0:4]
+            app_name = "ticketfrei" + str(db.get_secret())[0:4]
             client_id, client_secret \
                 = Mastodon.create_app(app_name, api_base_url=instance)
             db.execute("""INSERT INTO mastodon_instances(
@@ -361,10 +371,13 @@ Aber je mehr Leute mitmachen, desto eher kannst du dir sicher
 sein, dass wir sie finden, bevor sie uns finden.
 
 Wenn du immer direkt gewarnt werden willst, kannst du auch die
-Benachrichtigungen über E-Mail oder Telegram aktivieren. Gib 
-einfach <a href="/city/mail/""" + city + """"/">hier</a> deine 
-E-Mail-Adresse an oder subscribe dem Telegram-Bot [@ticketfrei_""" + city + \
-                   "_bot](https://t.me/ticketfrei_" + city + """_bot) 
+Benachrichtigungen über E-Mail, Telegram, oder den Mastodon RSS 
+feed aktivieren. Entweder:
+* Gibt hier [deine E-Mail-Adresse an](/city/mail/""" + city + """)
+* Subscribe dem Telegram-Bot [@ticketfrei_""" + city + \
+"_bot](https://t.me/ticketfrei_" + city + """_bot)
+* oder subscribe dem RSS feed von [""" + city + """](""" + masto_link + \
+""".atom?replies=false&boosts=true)
 
 Also, wenn du weniger Glück hast, und der erste bist, der einen 
 Kontrolleur sieht:
@@ -388,9 +401,9 @@ mentioned, und gib an
 
 Zum Beispiel so:
 
-![Screenshot of writing a Toot](https://github.com/b3yond/ticketfrei/raw/master/guides/tooting_screenshot.png)
+![Screenshot of writing a Toot](https://github.com/b3yond/ticketfrei/raw/stable1/guides/tooting_screenshot.png)
 
-![A toot ready to be shared](https://github.com/b3yond/ticketfrei/raw/master/guides/toot_screenshot.png)
+![A toot ready to be shared](https://github.com/b3yond/ticketfrei/raw/stable1/guides/toot_screenshot.png)
 
 Der Bot wird die Nachricht dann weiterverbreiten, auch zu den 
 anderen Netzwerken.