Batch Gmail filter creation CLI - watermint toolbox
28th July 2020New Gmail-related commands have been added to the watermint toolbox. This allows you to get a list of emails from the command line, add/remove/rename labels, and add/remove filters.
% tbx services google mail label list
watermint toolbox 72.4.544
==========================
© 2016-2020 Takayuki Okazaki
Licensed under open source licenses. Use the `license` command for more detail.
Testing network connection...
Done
Opening the authorization URL:
https://accounts.google.com/o/oauth2/auth?client_id=xxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A7800%2Fconnect%2Fauth&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fgmail.labels&state=xxxxxxxx
Please press ENTER to open the auth page on the browser.
| name | type |
|------------------------------|--------|
| CHAT | system |
| SENT | system |
| INBOX | system |
| IMPORTANT | system |
| TRASH | system |
| DRAFT | system |
| SPAM | system |
| CATEGORY_FORUMS | system |
| CATEGORY_UPDATES | system |
| CATEGORY_PERSONAL | system |
| CATEGORY_PROMOTIONS | system |
| CATEGORY_SOCIAL | system |
| STARRED | system |
| UNREAD | system |
| services/google.com | user |
| services/accounts.google.com | user |
| services/youtube.com | user |
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200727-074430.001/report/labels.csv
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200727-074430.001/report/labels.json
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200727-074430.001/report/labels.xlsx
If you want to use it, please use Release 72 or later.
I label and manage each source to some extent as I process a variety of emails. This is also a bit of a hassle when the number of source types increases, and I counted the number of labels in the command I created this time, and there are already 343 labels.
% tbx services google mail label list -output json | jq 'select(.type == "user") | .id ' | wc -l
343
I checked the number of filters created in the same way and found 474.
% tbx services google mail filter list -output json | wc -l
474
I have been making labels and filters by hand for several years now, but it’s getting tedious, so I made it possible to create labels and filters in batches based on certain conditions. So far, it’s not fully automated, but semi-automated once the CSV data file is created.
Create the source data for the label/filter you want to create.
There are two types of Gmail labels. (1) system labels that are automatically assigned by Gmail, such as INBOX and SPAM, and (2) user-defined user labels. In this time, I’m going to get the list of emails in INBOX and get the data with no user labels attached. In this case, I will add label conditions based on the destination address and source address of the email.
First of all, get the email data, get the source address, and make rules for each domain. I’ve omitted quite a bit, but I get the mail data in JSON format, process it with the jq command and output it as CSV. The first column is the query, the second column is the label to be added, and the third column is the label to be removed. The third column is not specified. If you want to archive automatically, you can do so by deleting INBOX.
% tbx services google mail message processed list -output json | jq -r 'select(.label_type_user | length ==0) | .from.address | ["from:"+., "services/"+capture("@(?<d>\\b([a-z0-9]+(-[a-z0-9]+)*\\.)+[a-z]{2,}\\b)").d] | @csv' | sort -u
"from:families-noreply@google.com","services/google.com"
"from:googlecommunityteam-noreply@google.com","services/google.com"
"from:no-reply@accounts.google.com","services/accounts.google.com"
"from:no-reply@google.com","services/google.com"
"from:noreply-purchases@youtube.com","services/youtube.com"
Create a batch filter based on this data. It also automatically creates any missing labels and applies them to the messages in the INBOX.
% tbx services google mail filter batch add -add-label-if-not-exist -apply-to-inbox-messages -peer waterlandpier -file ~/filters.csv
watermint toolbox `dev`
=======================
© 2016-2020 Takayuki Okazaki
Licensed under open source licenses. Use the `license` command for more detail.
Creating a filter: from:families-noreply@google.com
from:families-noreply@google.com: Updating message 1
from:families-noreply@google.com: Updating message 2
Creating a filter: from:googlecommunityteam-noreply@google.com
from:googlecommunityteam-noreply@google.com: Updating message 1
Creating a filter: from:no-reply@accounts.google.com
from:no-reply@accounts.google.com: Updating message 1
from:no-reply@accounts.google.com: Updating message 2
from:no-reply@accounts.google.com: Updating message 3
from:no-reply@accounts.google.com: Updating message 4
from:no-reply@accounts.google.com: Updating message 5
from:no-reply@accounts.google.com: Updating message 6
from:no-reply@accounts.google.com: Updating message 7
Creating a filter: from:no-reply@google.com
Creating a filter: from:noreply-purchases@youtube.com
from:noreply-purchases@youtube.com: Updating message 1
| status | reason | input.query | input.add_labels | input.delete_labels | result.id | result.criteria_from | result.criteria_to | result.criteria_subject | result.criteria_query | result.criteria_negated_query |
|--------|--------|---------------------------------------------|------------------------------|---------------------|----------------------------------------|----------------------|--------------------|-------------------------|---------------------------------------------|-------------------------------|
| 成功 | | from:families-noreply@google.com | services/google.com | | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | | | from:families-noreply@google.com | |
| 成功 | | from:googlecommunityteam-noreply@google.com | services/google.com | | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | | | from:googlecommunityteam-noreply@google.com | |
| 成功 | | from:no-reply@accounts.google.com | services/accounts.google.com | | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | | | from:no-reply@accounts.google.com | |
| 成功 | | from:no-reply@google.com | services/google.com | | xxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxx | | | | from:no-reply@google.com | |
| 成功 | | from:noreply-purchases@youtube.com | services/youtube.com | | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | | | from:noreply-purchases@youtube.com | |
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/filters.csv
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/filters.json
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/filters.xlsx
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/messages.csv
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/messages.json
The report generated: /xxxxx/xxxxxxxx/.toolbox/jobs/20200726-223228.001/report/messages.xlsx
I don’t usually have time to create filters in batches, so I’m hoping this will make my email processing a little more efficient.