Merge pull request #48 from b3yond/1.0

1.1

.gitignore

@@ -1,12 +1,17 @@
 *.swp
 *.pyc
 .idea/
+__pycache__/
 last_mention
+last_mail
 ticketfrei.cfg
+ticketfrei.sqlite
 seen_toots.pickle
 seen_toots.pickle.part
 pip-selfcheck.json
+config.toml
 bin/
 include/
 lib/
 share/
+local/

LICENSE

@@ -0,0 +1,14 @@
+Copyright (c) 2017 Thomas L <tom@dl6tom.de>
+Copyright (c) 2017 b3yond <b3yond@riseup.net>
+
+Permission to use, copy, modify, and distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

README.md

@@ -1,68 +1,96 @@
 # Ticketfrei micro messaging bot
 
+Version: 1.0
+
 <!-- This mastodon/twitter bot has one purpose - breaking the law. -->
 
-The functionality is simple: it retweets every tweet where it is mentioned.
+The functionality is simple: it retweets every tweet where it is
+mentioned.
 
-This leads to a community which evolves around it; if you see ticket controllers, you tweet their location and mention the bot. The bot then retweets your tweet and others can read the info and think twice if they want to buy a ticket. If enough people, a critical mass, participate for the bot to become reliable, you have positive self-reinforcing dynamics.
+This leads to a community which evolves around it; if you see ticket
+controllers, you tweet their location and mention the bot. The bot
+then retweets your tweet and others can read the info and think twice
+if they want to buy a ticket. If enough people, a critical mass,
+participate for the bot to become reliable, you have positive
+self-reinforcing dynamics.
 
-There is one security hole: people could start mentioning the bot with useless information, turning it into a spammer. That's why it has to be maintained; if someone spams the bot, mute them and undo the retweet. So it won't retweet their future tweets and the useless retweet is deleted if someone tries to check if something was retweeted in the last hour or something.
+In the promotion folder, you will find some promotion material you
+can use to build up such a community in your city. It is in german
+though =/
 
-Website: https://wiki.links-it.de/IT/Ticketfrei
+Website: https://wiki.links-tech.org/IT/Ticketfrei
 
-# Install
+## Install
 
-Install python and virtualenv with your favourite package manager.
-Create and activate virtualenv
+Setting up a ticketfrei bot for your city is quite easy. Here are the
+few steps:
+
+First you need to install python3 and virtualenv with your favourite
+package manager.
+
+Create and activate virtualenv:
 
 ```shell
-$ virtualenv -p python2 .
-$ . bin/activate
+sudo apt install python3 virtualenv
+virtualenv -p python3 .
+. bin/activate
 ```
-Install dependencies
+
+Install the dependencies:
 ```shell
-$ pip install python-twitter pytoml requests Mastodon.py
+pip install tweepy pytoml requests Mastodon.py
 ```
-Configure
+
+Configure the bot:
 ```shell
-$ cp ticketfrei.cfg.example ticketfrei.cfg
-$ vim ticketfrei.cfg
+cp config.toml.example config.toml
+vim config.toml
 ```
-Edit the account credentials, so your bot can use your accounts.
 
-Also add the words to the goodlist, which you want to require. A tweet is only retweeted, if it contains at least one of them. If you want to RT everything, just add your account name.
+You can use a Twitter, a Mastodon, and Mail with the account. They
+will communicate with each other; if someone warns others via Mail,
+Twitter and Mastodon users will also see the message. And vice versa.
 
-There is also a blacklist, which you can use to automatically sort out malicious tweets. Be careful though, our filter can't read the intention with which a word was used. Maybe you wanted it there.
+You have to configure all of the accounts via config.toml; it should
+be fairly intuitive to enter the right values.
 
-Note that atm the good- & blacklist are still outside of ticketfrei.cfg, in separate files. we will repare this soon.
+## Maintaining
 
-To keep the bots running when you are logged out of the shell, you can use screen:
+There is one security hole: people could start mentioning the bot
+with useless information, turning it into a spammer. That's why it
+has to be maintained; if someone spams the bot, mute them and undo
+the retweet. So it won't retweet their future tweets and the useless
+retweet is deleted if someone tries to check if something was
+retweeted in the last hour or something.
+
+To this date, we have never heard of this happening though.
+
+### blacklisting
+
+You also need to edit the goodlist and the blacklist. They are in the
+"goodlists" and "blacklists" folders. All text files in those
+directories will be used, so you should delete our templates; but
+feel free to use them as an orientation.
+
+Just add the words to the goodlist, which you want to require. A
+report is only spread, if it contains at least one of them. If you
+want to RT everything, just add a ```*```.
+
+There is also a blacklist, which you can use to automatically sort
+out malicious tweets. Be careful though, our filter can't read the
+intention with which a word was used. Maybe you wanted it there.
+
+### screen
+
+To keep the bots running when you are logged out of the shell, you
+can use screen:
 
 ```shell
 sudo apt-get install screen 
+echo "if [ -z "$STY" ]; then screen -RR; fi" >> ~/.bash_login
 screen
-python ticketfrei.py
+python3 ticketfrei.py
 ```
 
-## ideas
+To log out of the screen session, press "ctrl+a", and then "d".
 
-* You can only use the twitter API if you have confirmed a phone number and sacrificed a penguin in a blood ritual. So we should build it in a way that it uses the twitter web GUI. It's difficult, but maybe it works. We had another twitter bot that worked similarly, years ago: https://github.com/b3yond/twitter-bot
-* Build a tool that deletes wrong toots/tweets on both platforms, would work nicely with a web UI.
-* write the muted people to the db, to easily undo the mutes if necessary.
-
-## to do
-Desktop/pycharm-community-2017.1.4/bin/pycharm.sh
-- [x] Twitter: Crawl mentions
-- [x] Mastodon: Crawl mentions
-- [ ] Write toots/tweets to database/log
-- [x] Twitter: retweet people
-- [x] Mastodon: boost people
-- [x] Twitter: access the API
-- [ ] Web UI that lets you easily delete toots/tweets per db id and mute the tweet author
-- [x] Write Bots as Classes to be easier implemented
-- [x] Create extra Class for the filter
-- [x] Put as much as possible into ticketfrei.cfg
-- [x] Make both bots run on their own *and* next to each other
-  - [x] implement trigger class in retootbot
-  - [x] read config in retweetbot
-- [x] put shutdown contact in ticketfrei.cfg

blacklists/nbg_blacklist

@@ -8,3 +8,6 @@ jude
 schwuchtel
 fag
 faggot
+nigger
+neger
+schlitz

config.toml.example

@@ -0,0 +1,48 @@
+[mapp]
+# The bot registers a mastodon app automatically to acquire OAuth keys.
+name = 'yourcity_ticketfrei'    # What you want the app to be called
+
+[muser]
+enabled = 'false'  # set to true if you want to use Mastodon
+email = 'youremail@server.tld'  # E-mail address of your Mastodon account
+password = 'yourpassword'       # Password of your Mastodon account
+server = 'yourmastodoninstance' # Instance where you have your Mastodon account
+
+[tapp]
+# You get those keys when you follow these steps:
+# https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens
+consumer_key = "your_consumer_key"
+consumer_secret = "your_consumer_secret"
+
+[tuser]
+enabled = 'false'  # set to true if you want to use Twitter
+# You get those keys when you follow these steps:
+# https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens
+access_token_key = "your_access_token_key"
+access_token_secret = "your_acces_token_secret"
+
+[mail]
+enabled = 'false'  # set to true if you want to use Mail notifications
+# This is the mail the bot uses to send emails.
+mailserver = "smtp.riseup.net"
+user = "ticketfrei"
+passphrase = "sup3rs3cur3"
+# If you want to receive crash reports (so you can restart the bot
+# when it breaks down), you should specify a contact email address:
+#contact = "your_mail@riseup.net"
+# Mailing list where you want to send warnings to
+list = "yourcity_ticketfrei@lists.links-tech.org"
+
+[logging]
+# The directory where logs should be stored.
+logpath = "logs/ticketfrei.log"
+
+# [trigger]
+# goodlists are one regex per line.
+# badlists are one badword per line.
+# a message musst match at least one regex in goodlist and contain none of the badwords.
+# the variables mention the directory where the lists are located, not the filenames.
+# These are the default folders. If you want to specify differents folders, uncomment
+# those lines and enter relative paths.
+#goodlist_path = 'goodlists'
+#blacklist_path = 'blacklists'

local

@@ -1 +0,0 @@
-Subproject commit c8e9d7fd7ae0fe04921fdcf85e11fc9c0c324958

mailbot.py

@@ -0,0 +1,211 @@
+#!/usr/bin/env python3
+
+import sendmail
+import ssl
+import time
+import trigger
+import datetime
+import email
+import logging
+import pytoml as toml
+import imaplib
+import report
+
+logger = logging.getLogger(__name__)
+
+
+class Mailbot(object):
+    """
+    Bot which sends Mails if mentioned via twitter/mastodon, and tells
+    other bots that it received mails.
+    """
+
+    def __init__(self, config, history_path="last_mail"):
+        """
+        Creates a Bot who listens to mails and forwards them to other
+        bots.
+
+        :param config: (dictionary) config.toml as a dictionary of dictionaries
+        """
+        self.config = config
+
+        self.history_path = history_path
+        self.last_mail = self.get_history(self.history_path)
+
+        try:
+            self.mailinglist = self.config["mail"]["list"]
+        except KeyError:
+            self.mailinglist = None
+
+        self.mailbox = imaplib.IMAP4_SSL(self.config["mail"]["imapserver"])
+        context = ssl.create_default_context()
+        try:
+            self.mailbox.starttls(ssl_context=context)
+        except:
+            logger.error('StartTLS failed', exc_info=True)
+        try:
+            self.mailbox.login(self.config["mail"]["user"], self.config["mail"]["passphrase"])
+        except imaplib.IMAP4.error:
+            logger.error("Login to mail server failed", exc_info=True)
+            try:
+                mailer = sendmail.Mailer(config)
+                mailer.send('', config['mail']['contact'],
+                            'Ticketfrei Crash Report',
+                            attachment=config['logging']['logpath'])
+            except:
+                logger.error('Mail sending failed', exc_info=True)
+
+    def repost(self, status):
+        """
+        E-Mails don't have to be reposted - they already reached everyone on the mailing list.
+        The function still needs to be here because ticketfrei.py assumes it, and take the
+        report object they want to give us.
+
+        :param status: (report.Report object)
+        """
+        pass
+
+    def crawl(self):
+        """
+        crawl for new mails.
+        :return: msgs: (list of report.Report objects)
+        """
+        try:
+            rv, data = self.mailbox.select("Inbox")
+        except imaplib.IMAP4.abort:
+            rv = "Crawling Mail failed"
+            logger.error(rv, exc_info=True)
+        except TimeoutError:
+            rv = "No Connection"
+            logger.error(rv, exc_info=True)
+        msgs = []
+        if rv == 'OK':
+            rv, data = self.mailbox.search(None, "ALL")
+            if rv != 'OK':
+                return msgs
+
+            for num in data[0].split():
+                rv, data = self.mailbox.fetch(num, '(RFC822)')
+                if rv != 'OK':
+                    logger.error("Couldn't fetch mail %s %s" % (rv, str(data)))
+                    return msgs
+                msg = email.message_from_bytes(data[0][1])
+
+                if not self.config['mail']['user'] + "@" + \
+                        self.config["mail"]["mailserver"].partition(".")[2] in msg['From']:
+                    # get a comparable date out of the email
+                    date_tuple = email.utils.parsedate_tz(msg['Date'])
+                    date_tuple = datetime.datetime.fromtimestamp(email.utils.mktime_tz(date_tuple))
+                    date = (date_tuple - datetime.datetime(1970, 1, 1)).total_seconds()
+                    if date > self.get_history(self.history_path):
+                        self.last_mail = date
+                        self.save_last()
+                        msgs.append(self.make_report(msg))
+        return msgs
+
+    def get_history(self, path):
+        """
+        This counter is needed to keep track of your mails, so you
+        don't double parse them
+
+        :param path: string: contains path to the file where the ID of the
+            last_mail is stored.
+        :return: last_mail: ID of the last mail the bot parsed
+        """
+        try:
+            with open(path, "r+") as f:
+                last_mail = f.read()
+        except IOError:
+            with open(path, "w+") as f:
+                last_mail = "0"
+                f.write(last_mail)
+        return float(last_mail)
+
+    def save_last(self):
+        """ Saves the last retweeted tweet in last_mention. """
+        with open(self.history_path, "w") as f:
+            f.write(str(self.last_mail))
+
+    def post(self, status):
+        """
+        sends reports by other sources to a mailing list.
+
+        :param status: (report.Report object)
+        """
+        mailer = sendmail.Mailer(self.config)
+        mailer.send(status.format(), self.mailinglist, "Warnung: Kontrolleure gesehen")
+
+    def make_report(self, msg):
+        """
+        generates a report out of a mail
+
+        :param msg: email.parser.Message object
+        :return: post: report.Report object
+        """
+        # get a comparable date out of the email
+        date_tuple = email.utils.parsedate_tz(msg['Date'])
+        date_tuple = datetime.datetime.fromtimestamp(email.utils.mktime_tz(date_tuple))
+        date = (date_tuple-datetime.datetime(1970,1,1)).total_seconds()
+
+        author = msg.get("From")  # get mail author from email header
+        # :todo take only the part before the @
+
+        text = msg.get_payload()
+        post = report.Report(author, "mail", text, None, date)
+        self.last_mail = date
+        self.save_last()
+        return post
+
+    def flow(self, trigger, statuses):
+        """
+        to be iterated. uses trigger to separate the sheep from the goats
+
+        :param statuses: (list of report.Report objects)
+        :return: statuses: (list of report.Report objects)
+        """
+        for status in statuses:
+            self.post(status)
+
+        msgs = self.crawl()
+
+        statuses = []
+        for msg in msgs:
+            if trigger.is_ok(msg.get_payload()):
+                statuses.append(msg)
+        return statuses
+
+
+if __name__ == "__main__":
+    # read config in TOML format (https://github.com/toml-lang/toml#toml)
+    with open('config.toml') as configfile:
+        config = toml.load(configfile)
+
+    # set log file
+    logger = logging.getLogger()
+    fh = logging.FileHandler(config['logging']['logpath'])
+    fh.setLevel(logging.DEBUG)
+    logger.addHandler(fh)
+
+    # initialise trigger
+    trigger = trigger.Trigger(config)
+
+    # initialise mail bot
+    m = Mailbot(config)
+
+    statuses = []
+    try:
+        while 1:
+            print("Received Reports: " + str(m.flow(trigger, statuses)))
+            time.sleep(1)
+    except KeyboardInterrupt:
+        print("Good bye. Remember to restart the bot!")
+    except:
+        logger.error('Shutdown', exc_info=True)
+        m.save_last()
+        try:
+            mailer = sendmail.Mailer(config)
+            mailer.send('', config['mail']['contact'],
+                        'Ticketfrei Crash Report',
+                        attachment=config['logging']['logpath'])
+        except:
+            logger.error('Mail sending failed', exc_info=True)

promotion/campaign.md

@@ -0,0 +1,45 @@
+# Campaign to build a local community around ticketfrei
+
+## Target groups
+
+Students: usually already have a ticket, but may be solidaric
+* especially design university
+
+Leftist scene
+* Flyers in alternative centers
+* Graffitis in alternative neighbourhoods
+
+Schools: 
+* especially trade schools
+
+Nightlife
+* Spread flyers in bars and nightclubs
+
+Fare Dodger
+* ppl in the queue of the Service Center
+
+## Material
+
+Logo + Header Picture
+
+Sticker
+
+Flyer
+* 1 Flyer in high-polish VAG-Layout
+* 1 Flyer in DIY-Anarcho-Style
+
+Graffiti stencils
+
+## Video
+
+2-3 Minutes explaining video with Anonymous-Mask und cryptical language
+* Have fun with a greenscreen - a mask, floating through a populated subway
+
+Short video how to set up your own Ticketfrei bot
+
+## Talk in alternative centers
+
+talk, maybe 15 minutes
+* How does the bot work?
+* Why is this politically relevant, what is sousveillance?
+* What is Ticketfreier ÖPNV & why is it good for everyone?

promotion/flyer.md

@@ -1,3 +1,5 @@
+Flyer, der bestimmt gut an Universitäten geht:
+
 # Ticketfrei fahren? Nur wenn wir zusammenhelfen!
 
 Das Problem sind immer die Kontrolleure.

promotion/flyer_polite.md

@@ -0,0 +1,148 @@
+# Neues Konzept für ein Sozialticket - wie können auch sie Ticketfrei fahren?
+
+Die VAG stellt ein neues Konzept als Alternative zu unerwünschten
+Fahrkartenkontrollen vor. Immer mehr Beschwerden über aufdringliche
+Kontrolleure und Kontrollerinnen erreichen uns; und weil wir Wert auf
+Userfreundlichkeit legen, haben wir ein neues Konzept entwickelt, das
+Ihnen helfen soll, Fahrkartenkontrollen zu vermeiden. Ganz im Sinne
+der Digitalisierung nutzen wir Soziale Medien, um Ihr Leben zu
+erleichtern - ein Twitterbot soll Ihnen dabei helfen, Kontrollen
+zu umgehen.
+
+Sie als Fahrgast können davon profitieren, indem Sie einen Blick auf
+das @nbg_ticketfrei-Twitterprofil werfen. Dort tragen andere User und
+Userinnen Informationen darüber zusammen, wo gerade Kontrolleure
+unterwegs sind.
+
+Wenn auf einer Linie, die Sie benutzen wollen, gerade Kontrolleure
+und Kontrolleurinnen gesichtet wurden, kaufen Sie besser eine
+Fahrkarte. Wenn die Luft aber rein ist, können Sie sich die Kosten
+sparen und unser Angebot auch ohne Ticket nutzen. Wenn Sie viel
+unterwegs sind, sollten sie auch in Erwägung ziehen, unsere
+E-Mail-Notifications zu abonnieren.
+
+Wenn Sie als User oder Userin zu dem Konzept beitragen wollen, geht
+das ganz einfach - immer, wenn Sie Kontrollpersonen bei ihrer Arbeit
+sehen, schreiben Sie einfach einen Tweet an @nbg_ticketfrei, wo und
+in welcher Richtung diese gerade unterwegs sind. Unser Bot retweetet
+das dann, damit auch alle anderen über die Bedrohung Bescheid wissen.
+
+Damit wollen wir dass ÖPNV endlich für alle möglich wird. Zu viele
+Menschen können sich VAG-Tickets leider nicht leisten - gerade die,
+die ihre MobiCard am Ende des Monats kaufen, müssen oft zwischen der
+MobiCard und dem letzten Wocheneinkauf abwägen. Damit endlich alle
+Menschen in Nürnberg/Fürth/Erlangen U-Bahn fahren können, soll
+@nbg_ticketfrei ihnen helfen, Kontrollen zu vermeiden.
+
+Weitere Informationen, wie Sie mitmachen können, finden Sie unter
+[Wiki-Page auf Deutsch, Englisch, Türkisch, Russisch, Spanisch]
+
+[QR-Code]
+
+
+# Was sagen die Menschen dazu?
+
+Melina Moliescu, KFZ-Mechanikerin (28): "Bevor es Ticketfrei gab,
+war es immer ein Glücksspiel, ohne Ticket mit der VAG unterwegs zu
+sein. Ich musste einen großen Teil meines Lohns entweder für teure
+Einzelfahrkarten ausgeben oder für Bußgelder - wenn ich mal kein
+Glück hatte. Jetzt sehe ich fast immer, wann ich mir ein Ticket
+kaufen sollte, und wann ich es mir sparen kann. Durch das neue
+Ticketfrei-System habe ich endlich genug Geld auf die Seite legen
+können, um mal wieder in den Urlaub zu fliegen!"
+
+Adolf Eichmann, Kontrolleur (45): "Dank Ticketfrei komme ich nun viel
+seltener in die unangenehme Situation, Menschen wegen eines fehlenden
+Fahrscheins belästigen zu müssen. Das macht meinen Job bedeutend
+entspannter! Ich kontrolliere jetzt wirklich nur noch Leute, die auch
+kontrolliert werden wollen. Niemand spuckt mich in der U-Bahn mehr an
+oder beleidigt mich. Meine Familie findet auch, dass ich Abends viel
+relaxter bin als früher, meine Tochter fängt langsam wieder an,
+Vertrauen zu mir zu fassen."
+
+Tick, Trick & Track, Schüler (alle 12): "In unsere Schülertickets
+müssen wir jeden Monat die neue Monatskarte einlegen und eine
+sechsstellige Nummer abschreiben - unnötig kompliziert, wir
+vergessen das jedes Mal. Und unsere Freunde und Freundinnen in
+Erlangen können wir damit auch nicht besuchen. Mit Ticketfrei
+kann man die Kontolleure und Kontrolleurinnen weitläufig umgehen,
+statt plötzlich hastig davonwatscheln zu müssen."
+
+Mehmet Müller, Kindergärtner (34): "Ich finde es schade, dass nicht
+alle Menschen sich U-Bahnen leisten können. Mit meinem sozialen
+Beruf wäre das auch schwierig, zum Glück verdient meine Frau genug,
+darum gönnen wir uns dieses Privileg. Aber ich sehe nicht ein, warum
+man die U-Bahn dann nicht auch voll packen soll! Bringt doch nichts,
+wenn manche auf dem Bahnsteig stehen bleiben und laufen müssen, wenn
+noch Platz in der Bahn wäre. Sie fährt doch bereits, ich und die
+anderen haben ja bezahlt. Warum sollen die, die nicht bezahlen
+können, nicht mitfahren?"
+
+
+## Slogan für 1 Werbeanzeige oder so:
+
+WENN Sie Ihre Ziele jeden Tag risikolos & kostenfrei erreichen,
+DANN nur weil sie den Ticketfrei-Service der VGN nutzen und nicht
+auf gut Glück schwarzfahren.
+
+WENN
+HEIMAT scheiße ist,
+DANN ist
+SÖDER dafür Verantwortlich
+(eine Anzeige von Ihrem Heimatministerium und verehrten Ministerpräsidenten (fast))
+
+WENN du beim
+SCHWARZFAHREN erwischt wirst,
+DANN nur weil Du noch nicht den Ticketfrei-Service deiner
+VGN nutzt.
+
+WENN
+ÜBERWACHUNG sich lohnt,
+DANN weil die
+VAG öffentlich ist
+
+# Eine Zukunft ohne Ticketautomaten und Kontrolleure
+
+Ticketfrei ist nicht nur ein Ansatz für Userfreundlichkeit, es ist
+gleichzeitig ein Modellprojekt für die Zukunft. Ticketfreier ÖPNV
+ist ein unkomplizierter Ansatz, Mobilität für alle zu finanzieren.
+
+Dabei werden Busse und Bahnen vollständig aus Steuern bezahlt. Das
+derzeitige Ticketsystem ist unnötig kompliziert - oft ist es unklar,
+welche Tickets wo gelten, wie lange, und was man für sein Geld
+bekommt. Wenn der ÖPNV komplett durch Steuern finanziert wäre,
+bräuchte man keine Kontrolleure und Kontrolleurinnen, keine Automaten
+und deren Wartung mehr zu finanzieren - und Ihr Geld wird direkt
+dafür benutzt, Sie und alle anderen von A nach B zu bringen.
+
+Wenn man die U-Bahn sozusagen bereits bezahlt hat, gibt es auch
+weniger Gründe, im Alltag noch mit dem Auto zu fahren. Es ist nicht
+nur gut für die Umwelt, wenn mehr Leute die Bahnen nutzen. Es
+entlastet auch die Innenstädte, weniger Smog, Lärm, Hektik, und
+weniger Verkehrsunfälle tragen zu einem guten Miteinander bei.
+
+Das ist auch für Leute aus dem Umland gut: kein Verkehrschaos und
+keine nervenaufreibende Parkplatzsuche in der Innenstadt mehr. Man
+kann einfach am Stadtrand parken und entspannt die U-Bahn nehmen, um
+seine Einkäufe zu tätigen und Verwandte zu besuchen.
+
+Für Touristen bedeutet der ticketfreie ÖPNV eine massive
+Erleichterung. In einer fremden Stadt herauszufinden, welche
+Preisstufe man bezahlen muss, ist für viele schwierig. Gerade wenn
+man kein Deutsch spricht, ist das komplexe System der Tarifzonen,
+Kurzstrecken, und Gültigkeitsdauer schwer zu begreifen. Mit dem
+falschen Ticket erwischt zu werden, kann einem den Urlaub schon mal
+versauen.
+
+Stattdessen kann Nürnberg als die Stadt bekannt werden, in der
+U-Bahnen einen kostenlos überall hin mitnehmen. Tourismuseinnahmen
+kommen der breiten Bevölkerung zu Gute. Neben den fahrerlosen
+U-Bahnen, dem Doku-Zentrum, der Burg, den Nürnberger Prozessen und
+natürlich dem Christkindelsmarkt kann ticketfreier ÖPNV dazu
+beitragen, Nürnberg als Weltstadt bekannt zu machen - hier wird
+nicht nur Geschichte, sondern auch die Zukunft geschrieben.
+
+Denn in der Zukunft fahren die U-Bahnen nicht nur fahrerlos, sondern
+auch ohne Tickets!
+
+

report.py

@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+
+class Report(object):
+    """
+    A ticketfrei report object.
+
+    Toots, Tweets, and E-Mails can be formed into ticketfrei reports.
+
+    """
+
+    def __init__(self, author, source, text, id, timestamp):
+        """
+        Constructor of a ticketfrei report
+
+        :param author: username of the author
+        :param source: mastodon, twitter, or email
+        :param text: the text of the report
+        :param id: id in the network
+        :param timestamp: time of the report
+        """
+        self.author = author
+        self.type = source
+        self.text = text
+        self.timestamp = timestamp
+        self.id = id
+
+    def format(self):
+        """
+        Format the report for bot.post()
+
+        :rtype: string
+        :return: toot: text to be tooted, e.g. "_b3yond: There are
+            uniformed controllers in the U2 at Opernhaus."
+        """
+        strng = self.author + ": " + self.text
+        return strng

retootbot.py

@@ -6,17 +6,19 @@ import os
 import pickle
 import re
 import time
-import datetime
 import trigger
-import traceback
+import logging
+import sendmail
+import report
+
+logger = logging.getLogger(__name__)
 
 
 class RetootBot(object):
-    def __init__(self, config, filter, logpath=None):
+    def __init__(self, config):
         self.config = config
-        self.filter = filter
-        self.register()
-        self.login()
+        self.client_id = self.register()
+        self.m = self.login()
 
         # load state
         try:
@@ -25,85 +27,102 @@ class RetootBot(object):
         except IOError:
             self.seen_toots = set()
 
-        if logpath:
-            self.logpath = logpath
-        else:
-            self.logpath = os.path.join("logs", str(datetime.datetime.now()))
-
-    def log(self, message, tb=False):
-        """
-        Writing an error message to a logfile in logs/ and prints it.
-
-        :param message(string): Log message to be displayed
-        :param tb: String of the Traceback
-        """
-        time = str(datetime.datetime.now())
-        if tb:
-            message = message + " The traceback is located at " + os.path.join("logs" + time)
-            with open(os.path.join("logs", time), 'w+') as f:
-                f.write(tb)
-        line = "[" + time + "] "+ message + "\n"
-        with open(self.logpath, 'a') as f:
-            try:
-                f.write(line)
-            except UnicodeEncodeError:
-                self.log("Failed to save log message due to UTF-8 error. ")
-                traceback.print_exc()
-        print line,
-
     def register(self):
-        self.client_id = os.path.join(
-                'appkeys',
-                self.config['mapp']['name'] +
-                '@' + self.config['muser']['server']
-            )
+        client_id = os.path.join(
+            'appkeys',
+            self.config['mapp']['name'] +
+            '@' + self.config['muser']['server']
+        )
 
-        if not os.path.isfile(self.client_id):
+        if not os.path.isfile(client_id):
             mastodon.Mastodon.create_app(
-                    self.config['mapp']['name'],
-                    api_base_url=self.config['muser']['server'],
-                    to_file=self.client_id
-                )
+                self.config['mapp']['name'],
+                api_base_url=self.config['muser']['server'],
+                to_file=client_id
+            )
+        return client_id
 
     def login(self):
-        self.m = mastodon.Mastodon(
-                client_id=self.client_id,
-                api_base_url=self.config['muser']['server']
-            )
-        self.m.log_in(
-                self.config['muser']['email'],
-                self.config['muser']['password']
-            )
+        m = mastodon.Mastodon(
+            client_id=self.client_id,
+            api_base_url=self.config['muser']['server']
+        )
+        m.log_in(
+            self.config['muser']['email'],
+            self.config['muser']['password']
+        )
+        return m
 
-    def retoot(self, toots=()):
+    def save_last(self):
+        """ save the last seen toot """
+        try:
+            with os.fdopen(os.open('seen_toots.pickle.part', os.O_WRONLY | os.O_EXCL | os.O_CREAT), 'wb') as f:
+                pickle.dump(self.seen_toots, f)
+        except FileExistsError:
+            os.unlink('seen_toots.pickle.part')
+            with os.fdopen(os.open('seen_toots.pickle.part', os.O_WRONLY | os.O_EXCL | os.O_CREAT), 'wb') as f:
+                pickle.dump(self.seen_toots, f)
+        os.rename('seen_toots.pickle.part', 'seen_toots.pickle')
+
+    def crawl(self):
+        """
+        Crawl mentions from Mastodon.
+
+        :return: list of statuses
+        """
+        mentions = []
+        try:
+            all = self.m.notifications()
+        except:  # mastodon.Mastodon.MastodonAPIError is unfortunately not in __init__.py
+            logger.error("Unknown Mastodon API Error.", exc_info=True)
+            return mentions
+        for status in all:
+            if (status['type'] == 'mention' and status['status']['id'] not in self.seen_toots):
+                # save state
+                self.seen_toots.add(status['status']['id'])
+                self.save_last()
+                # add mention to mentions
+                text = re.sub(r'<[^>]*>', '', status['status']['content'])
+                text = re.sub("(?<=^|(?<=[^a-zA-Z0-9-_\.]))@([A-Za-z]+[A-Za-z0-9-_]+)", "", text)
+                mentions.append(report.Report(status['account']['acct'],
+                                              "mastodon",
+                                              text,
+                                              status['status']['id'],
+                                              status['status']['created_at']))
+        return mentions
+
+    def repost(self, mention):
+        """
+        Retoots a mention.
+
+        :param mention: (report.Report object)
+        """
+        logger.info('Boosting toot from %s' % (
+            mention.format()))
+        self.m.status_reblog(mention.id)
+
+
+    def post(self, report):
+        """
+        Toots a report from other sources.
+
+        :param report: (report.Report object)
+        """
+        toot = report.format()
+        self.m.toot(toot)
+
+    def flow(self, trigger, reports=()):
         # toot external provided messages
-        for toot in toots:
-            self.m.toot(toot)
+        for report in reports:
+            self.post(report)
 
         # boost mentions
         retoots = []
-        for notification in self.m.notifications():
-            if (notification['type'] == 'mention'
-                    and notification['status']['id'] not in self.seen_toots):
-                self.seen_toots.add(notification['status']['id'])
-                text_content = re.sub(r'<[^>]*>', '',
-                                      notification['status']['content'])
-                if not self.filter.is_ok(text_content):
-                    continue
-                self.log('Boosting toot from %s: %s' % (
-                    #notification['status']['id'],
-                    notification['status']['account']['acct'],
-                    notification['status']['content']))
-                self.m.status_reblog(notification['status']['id'])
-                retoots.append('%s: %s' % (
-                    notification['status']['account']['acct'],
-                    re.sub(r'@\S*', '', text_content)))
-        # If the Mastodon instance returns interesting Errors, add them here:
-
-        # save state
-        with os.fdopen(os.open('seen_toots.pickle.part', os.O_WRONLY | os.O_EXCL | os.O_CREAT), 'w') as f:
-            pickle.dump(self.seen_toots, f)
-        os.rename('seen_toots.pickle.part', 'seen_toots.pickle')
+        for mention in self.crawl():
+            if not trigger.is_ok(mention.text):
+                continue
+            self.repost(mention)
+            retoots.append(mention)
 
         # return mentions for mirroring
         return retoots
@@ -111,12 +130,28 @@ class RetootBot(object):
 
 if __name__ == '__main__':
     # read config in TOML format (https://github.com/toml-lang/toml#toml)
-    with open('ticketfrei.cfg') as configfile:
+    with open('config.toml') as configfile:
         config = toml.load(configfile)
 
-    filter = trigger.Trigger(config)
-    bot = RetootBot(config, filter)
+    fh = logging.FileHandler(config['logging']['logpath'])
+    fh.setLevel(logging.DEBUG)
+    logger.addHandler(fh)
 
-    while True:
-        bot.retoot()
-        time.sleep(1)
+    trigger = trigger.Trigger(config)
+    bot = RetootBot(config)
+
+    try:
+        while True:
+            bot.flow(trigger)
+            time.sleep(1)
+    except KeyboardInterrupt:
+            print("Good bye. Remember to restart the bot!")
+    except:
+        logger.error('Shutdown', exc_info=True)
+        try:
+            mailer = sendmail.Mailer(config)
+            mailer.send('', config['mail']['contact'],
+                        'Ticketfrei Crash Report',
+                        attachment=config['logging']['logpath'])
+        except:
+            logger.error('Mail sending failed', exc_info=True)

retweetbot.py

@@ -1,13 +1,16 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
-import twitter
-import os
-import datetime
+import tweepy
+import re
 import requests
 import pytoml as toml
 import trigger
 from time import sleep
-import traceback
+import report
+import logging
+import sendmail
+
+logger = logging.getLogger(__name__)
 
 
 class RetweetBot(object):
@@ -18,47 +21,36 @@ class RetweetBot(object):
 
     api: The api object, generated with your oAuth keys, responsible for
         communication with twitter rest API
-    triggers: a list of words, one of them has to be in a tweet for it to be
-        retweeted
     last_mention: the ID of the last tweet which mentioned you
     """
 
-    def __init__(self, trigger, config, historypath="last_mention", logpath=None):
+    def __init__(self, config, history_path="last_mention"):
         """
         Initializes the bot and loads all the necessary data.
 
-        :param historypath: Path to the file with ID of the last retweeted
+        :param config: (dictionary) config.toml as a dictionary of dictionaries
+        :param history_path: Path to the file with ID of the last retweeted
             Tweet
-        :param logpath: Path to the file where the log is stored
         """
         self.config = config
+
+        # initialize API access
         keys = self.get_api_keys()
-        self.api = twitter.Api(consumer_key=keys[0],
-                               consumer_secret=keys[1],
-                               access_token_key=keys[2],
-                               access_token_secret=keys[3])
-        self.historypath = historypath
-        try:
-            self.no_shutdown_contact = False
-            self.user_id = self.config['tapp']['shutdown_contact_userid']
-            self.screen_name = \
-                self.config['tapp']['shutdown_contact_screen_name']
-        except KeyError:
-            self.no_shutdown_contact = True
-        self.last_mention = self.get_history(self.historypath)
-        self.trigger = trigger
+        auth = tweepy.OAuthHandler(consumer_key=keys[0],
+                                   consumer_secret=keys[1])
+        auth.set_access_token(keys[2],  # access_token_key
+                              keys[3])  # access_token_secret
+        self.api = tweepy.API(auth)
+
+        self.history_path = history_path
+        self.last_mention = self.get_history(self.history_path)
         self.waitcounter = 0
-        if logpath:
-            self.logpath = logpath
-        else:
-            self.logpath = os.path.join("logs", str(datetime.datetime.now()))
-        print "Path of logfile: " + self.logpath
 
     def get_api_keys(self):
         """
         How to get these keys is described in doc/twitter_api.md
 
-        After you received keys, store them in your ticketfrei.cfg like this:
+        After you received keys, store them in your config.toml like this:
         [tapp]
         consumer_key = "..."
         consumer_secret = "..."
@@ -69,34 +61,10 @@ class RetweetBot(object):
 
         :return: keys: list of these 4 strings.
         """
-        keys = []
-        keys.append(self.config['tapp']['consumer_key'])
-        keys.append(self.config['tapp']['consumer_secret'])
-        keys.append(self.config['tuser']['access_token_key'])
-        keys.append(self.config['tuser']['access_token_secret'])
+        keys = [self.config['tapp']['consumer_key'], self.config['tapp']['consumer_secret'],
+                self.config['tuser']['access_token_key'], self.config['tuser']['access_token_secret']]
         return keys
 
-    def log(self, message, tb=False):
-        """
-        Writing an error message to a logfile in logs/ and prints it.
-
-        :param message(string): Log message to be displayed
-        :param tb: String of the Traceback
-        """
-        time = str(datetime.datetime.now())
-        if tb:
-            message = message + " The traceback is located at " + os.path.join("logs" + time)
-            with open(os.path.join("logs", time), 'w+') as f:
-                f.write(tb)
-        line = "[" + time + "] "+ message + "\n"
-        with open(self.logpath, 'a') as f:
-            try:
-                f.write(line)
-            except UnicodeEncodeError:
-                self.log("Failed to save log message due to UTF-8 error. ")
-                traceback.print_exc()
-        print line,
-
     def get_history(self, path):
         """ This counter is needed to keep track of your mentions, so you
         don't double RT them
@@ -114,9 +82,9 @@ class RetweetBot(object):
                 f.write(last_mention)
         return int(last_mention)
 
-    def save_last_mention(self):
+    def save_last(self):
         """ Saves the last retweeted tweet in last_mention. """
-        with open(self.historypath, "w") as f:
+        with open(self.history_path, "w") as f:
             f.write(str(self.last_mention))
 
     def waiting(self):
@@ -130,141 +98,140 @@ class RetweetBot(object):
             self.waitcounter -= 1
         return self.waitcounter
 
-    def format_mastodon(self, status):
-        """
-        Bridge your Retweets to mastodon.
-        :todo vmann: add all the mastodon API magic.
-
-        :param status: Object of a tweet.
-        :return: toot: text tooted on mastodon, e.g. "_b3yond: There are
-            uniformed controllers in the U2 at Opernhaus."
-        """
-        toot = status.user.name + ": " + status.text
-        return toot
-
-    def crawl_mentions(self):
+    def crawl(self):
         """
         crawls all Tweets which mention the bot from the twitter rest API.
 
-        :return: list of Status objects
+        :return: reports: (list of report.Report objects)
         """
+        reports = []
         try:
             if not self.waiting():
-                mentions = self.api.GetMentions(since_id=self.last_mention)
-                return mentions
-        except twitter.TwitterError:
-            self.log("Twitter API Error: Rate Limit Exceeded.")
+                if self.last_mention == 0:
+                    mentions = self.api.mentions_timeline()
+                else:
+                    mentions = self.api.mentions_timeline(since_id=self.last_mention)
+                for status in mentions:
+                    text = re.sub("(?<=^|(?<=[^a-zA-Z0-9-_\.]))@([A-Za-z]+[A-Za-z0-9-_]+)", "", status.text)
+                    reports.append(report.Report(status.author.screen_name,
+                                                 "twitter",
+                                                 text,
+                                                 status.id,
+                                                 status.created_at))
+                self.save_last()
+                return reports
+        except tweepy.RateLimitError:
+            logger.error("Twitter API Error: Rate Limit Exceeded", exc_info=True)
             self.waitcounter += 60*15 + 1
         except requests.exceptions.ConnectionError:
-            self.log("Twitter API Error: Bad Connection.")
+            logger.error("Twitter API Error: Bad Connection", exc_info=True)
             self.waitcounter += 10
-        return None
+        except tweepy.TweepError:
+            logger.error("Twitter API Error: General Error", exc_info=True)
+        return []
 
-    def retweet(self, status):
+    def repost(self, status):
         """
         Retweets a given tweet.
 
-        :param status: A tweet object.
+        :param status: (report.Report object)
         :return: toot: string of the tweet, to toot on mastodon.
         """
         while 1:
             try:
-                self.api.PostRetweet(status.id)
-                self.log("Retweeted: " + self.format_mastodon(status))
+                self.api.retweet(status.id)
+                logger.info("Retweeted: " + status.format())
                 if status.id > self.last_mention:
                     self.last_mention = status.id
-                return self.format_mastodon(status)
-            # maybe one day we get rid of this error. If not, try to uncomment
-            # these lines.
-            except twitter.error.TwitterError:
-                self.log("Twitter API Error: You probably already retweeted this tweet: " + status.text)
-                if status.id > self.last_mention:
-                    self.last_mention = status.id
-                return None
+                self.save_last()
+                return status.format()
             except requests.exceptions.ConnectionError:
-                self.log("Twitter API Error: Bad Connection.")
+                logger.error("Twitter API Error: Bad Connection", exc_info=True)
                 sleep(10)
+            # maybe one day we get rid of this error:
+            except tweepy.TweepError:
+                logger.error("Twitter Error", exc_info=True)
+                if status.id > self.last_mention:
+                    self.last_mention = status.id
+                self.save_last()
+                return None
 
-    def tweet(self, post):
+    def post(self, status):
         """
         Tweet a post.
 
-        :param post: String with the text to tweet.
+        :param status: (report.Report object)
         """
-        if len(post) > 280:
-            post = post[:280 - 4] + u' ...'
+        text = status.format()
+        if len(text) > 280:
+            text = status.text[:280 - 4] + u' ...'
         while 1:
             try:
-                self.api.PostUpdate(status=post)
+                self.api.update_status(status=text)
                 return
             except requests.exceptions.ConnectionError:
-                self.log("Twitter API Error: Bad Connection.")
+                logger.error("Twitter API Error: Bad Connection", exc_info=True)
                 sleep(10)
 
-    def flow(self, to_tweet=()):
+    def flow(self, trigger, to_tweet=()):
         """ The flow of crawling mentions and retweeting them.
 
         :param to_tweet: list of strings to tweet
         :return list of retweeted tweets, to toot on mastodon
         """
 
-        # Tweet the toots the Retootbot gives to us
+        # Tweet the reports from other sources
         for post in to_tweet:
-            self.tweet(post)
+            self.post(post)
 
         # Store all mentions in a list of Status Objects
-        mentions = self.crawl_mentions()
-        mastodon = []
+        mentions = self.crawl()
 
-        if mentions is not None:
-            for status in mentions:
-                # Is the Text of the Tweet in the triggerlist?
-                if self.trigger.is_ok(status.text):
-                    # Retweet status
-                    toot = self.retweet(status)
-                    if toot:
-                        mastodon.append(toot)
+        # initialise list of strings for other bots
+        all_tweets = []
 
-                # save the id so it doesn't get crawled again
-                if status.id > self.last_mention:
-                    self.last_mention = status.id
-                self.save_last_mention()
-        # Return Retweets for tooting on mastodon
-        return mastodon
+        for status in mentions:
+            # Is the Text of the Tweet in the triggerlist?
+            if trigger.is_ok(status.text):
+                # Retweet status
+                toot = self.repost(status)
+                if toot:
+                    all_tweets.append(toot)
 
-    def shutdown(self):
-        """ If something breaks, it shuts down the bot and messages the owner.
-        """
-        logmessage = "Shit went wrong, closing down."
-        if self.screen_name:
-            logmessage = logmessage + " Sending message to " + self.screen_name
-        self.log(logmessage)
-        if self.no_shutdown_contact:
-            return
-        self.save_last_mention()
-        try:
-            self.api.PostDirectMessage("Help! I broke down. restart me pls :$",
-                                   self.user_id, self.screen_name)
-        except:
-            traceback.print_exc()
-            print
+        # Return Retweets for posting on other bots
+        return all_tweets
 
 
 if __name__ == "__main__":
-    # create an Api object
-    with open('ticketfrei.cfg') as configfile:
+    # get the config dict of dicts
+    with open('config.toml') as configfile:
         config = toml.load(configfile)
 
+    # set log file
+    fh = logging.FileHandler(config['logging']['logpath'])
+    fh.setLevel(logging.DEBUG)
+    logger.addHandler(fh)
+
+    # initialise trigger
     trigger = trigger.Trigger(config)
 
-    bot = RetweetBot(trigger, config)
+    # initialise twitter bot
+    bot = RetweetBot(config)
+
     try:
         while True:
-            bot.flow()
+            # :todo separate into small functions
+            bot.flow(trigger)
             sleep(60)
     except KeyboardInterrupt:
-        print "Good bye! Remember to restart the bot."
+        print("Good bye. Remember to restart the bot!")
     except:
-        traceback.print_exc()
-        print
-        bot.shutdown()
+        logger.error('Shutdown', exc_info=True)
+        bot.save_last()
+        try:
+            mailer = sendmail.Mailer(config)
+            mailer.send('', config['mail']['contact'],
+                        'Ticketfrei Crash Report',
+                        attachment=config['logging']['logpath'])
+        except:
+            logger.error('Mail sending failed', exc_info=True)

sendmail.py

@@ -0,0 +1,75 @@
+#!/usr/bin/env python3
+
+import smtplib
+import ssl
+import pytoml as toml
+from email.mime.text import MIMEText
+from email.mime.application import MIMEApplication
+from email.mime.multipart import MIMEMultipart
+
+
+class Mailer(object):
+    """
+    Maintains the connection to the mailserver and sends text to users.
+    """
+
+    def __init__(self, config):
+        """
+        Creates an SMTP client to send a mail. Is called only once
+        when you actually want to send a mail. After you sent the
+        mail, the SMTP client is shut down again.
+
+        :param config: The config file generated from config.toml
+        """
+        # This generates the From address by stripping the part until the first
+        # period from the mail server address and won't work always.
+        self.fromaddr = config["mail"]["user"] + "@" + \
+                        config["mail"]["mailserver"].partition(".")[2]
+
+        # starts a client session with the SMTP server
+        self.s = smtplib.SMTP(config["mail"]["mailserver"])
+        context = ssl.create_default_context()
+        self.s.starttls(context=context)
+        self.s.login(config["mail"]["user"], config["mail"]["passphrase"])
+
+    def send(self, text, recipient, subject, attachment=None):
+        """
+
+        :param text: (string) the content of the mail
+        :param recipient: (string) the recipient of the mail
+        :param subject: (string) the subject of the mail
+        :param attachment: (string) the path to the logfile
+        :return: string for logging purposes, contains recipient & subject
+        """
+        msg = MIMEMultipart()
+        msg.attach(MIMEText(text))
+
+        msg["From"] = self.fromaddr
+        msg["To"] = recipient
+        msg["Subject"] = subject
+
+        # attach logfile
+        if attachment:
+            with open(attachment, "rb") as fil:
+                part = MIMEApplication(
+                    fil.read(),
+                    Name="logfile"
+                )
+            # After the file is closed
+            part['Content-Disposition'] = 'attachment; filename="logfile"'
+            msg.attach(part)
+
+        self.s.send_message(msg)
+        self.s.close()
+
+        return "Sent mail to " + recipient + ": " + subject
+
+
+# For testing:
+if __name__ == '__main__':
+    # read config in TOML format (https://github.com/toml-lang/toml#toml)
+    with open('config.toml') as configfile:
+        config = toml.load(configfile)
+
+    m = Mailer(config)
+    print(m.send("This is a test mail.", m.fromaddr, "Test"))

ticketfrei.cfg.example

@@ -1,27 +0,0 @@
-[mapp]
-name = 'yourcity_ticketfrei'
-
-[muser]
-email = 'youremail@server.tld'
-password = 'yourpassword'
-server = 'yourmastodoninstance'
-
-[tapp]
-consumer_key = "yourconsumerkey"
-consumer_secret = yourconsumersecret"
-
-# shutdown_contact_userid = 012345
-# shutdown_contact_screen_name = 'yourscreenname'
-
-[tuser]
-access_token_key = "youraccesstokenkey"
-access_token_secret = "youraccesstokensecret"
-
-# [trigger]
-# goodlists are one regex per line.
-# badlists are one badword per line.
-# a message musst match at least one regex in goodlist and contain none of the badwords.
-# the variables mention the directory where the lists are located, not the filenames.
-
-# goodlist_path = 'goodlists'
-# blacklist_path = 'blacklists'

ticketfrei.py

@@ -1,36 +1,61 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
+import logging
 import pytoml as toml
 import time
-import traceback
-import os
-import datetime
+import sendmail
 
 from retootbot import RetootBot
 from retweetbot import RetweetBot
+from mailbot import Mailbot
 from trigger import Trigger
 
-
 if __name__ == '__main__':
     # read config in TOML format (https://github.com/toml-lang/toml#toml)
-    with open('ticketfrei.cfg') as configfile:
+    with open('config.toml') as configfile:
         config = toml.load(configfile)
 
+    # set log file
+    logger = logging.getLogger()
+    fh = logging.FileHandler(config['logging']['logpath'])
+    fh.setLevel(logging.DEBUG)
+    logger.addHandler(fh)
+
     trigger = Trigger(config)
 
-    logpath = os.path.join("logs", str(datetime.datetime.now()))
+    bots = []
 
-    mbot = RetootBot(config, trigger, logpath=logpath)
-    tbot = RetweetBot(trigger, config, logpath=logpath)
+    if config["muser"]["enabled"] != "false":
+        bots.append(RetootBot(config))
+    if config["tuser"]["enabled"] != "false":
+        bots.append(RetweetBot(config))
+    if config["mail"]["enabled"] != "false":
+        bots.append(Mailbot(config))
 
     try:
         statuses = []
         while True:
-            statuses = mbot.retoot(statuses)
-            statuses = tbot.flow(statuses)
-            time.sleep(60)
+            for bot in bots:
+                reports = bot.crawl()
+                for status in reports:
+                    if not trigger.is_ok(status.text):
+                        continue
+                    for bot2 in bots:
+                        if bot == bot2:
+                            bot2.repost(status)
+                        else:
+                            bot2.post(status)
+            time.sleep(60)  # twitter rate limit >.<
     except KeyboardInterrupt:
-        print "Good bye. Remember to restart the bot!"
+        print("Good bye. Remember to restart the bot!")
     except:
-        traceback.print_exc()
-        tbot.shutdown()
+        logger.error('Shutdown', exc_info=True)
+        for bot in bots:
+            bot.save_last()
+        try:
+            mailer = sendmail.Mailer(config)
+            mailer.send('', config['mail']['contact'],
+                        'Ticketfrei Crash Report',
+                        attachment=config['logging']['logpath'])
+        except:
+            logger.error('Mail sending failed', exc_info=True)

trigger.py

@@ -59,7 +59,7 @@ class Trigger(object):
 
 
 if __name__ == "__main__":
-    with open("ticketfrei.cfg", "r") as configfile:
+    with open("config.toml", "r") as configfile:
         config = toml.load(configfile)
 
     print("testing the trigger")