matrix-report-documentation/readme.md

FiftyFiftyOneArizona's Matrix-Based Reporting System

This file documents a currently under development system which collects and distributes ICE sightings in a secure, decentralized, and resilient way using Matrix.

At the time of writing, this system is still being planned and is far from reaching maturity. However, the first version of this document (and the first commit to the repository holding it) will contain a fully complete explanation on how the system works.

The decision to make the specification as rigid as it is comes from the knowledge that this system will be relied upon by people to not be kidnapped and illegally sent to an El Salvadorian gulag. However, the specification is rather simple and only aims to be a standardized way to collect and distribute information, with a heavy focus on real-word locations in the U.S.

Why Matrix

Matrix is more or less uncontested as a medium to distribute this information. Matrix has an already-built ecosystem of servers, clients, bots, and SDKs which lets people easily add in nearly any functionality they want. Matrix is also distributed, fault-tolerant, secured with SSL, and fully authenticated. We hope that more organizations will spin up their own Matrix server and contribute information to this system.

Event Format

Every event is a text message (m.text). This is done so that real people can contribute information directly to the system, and so that Maubot works with it nicely.

Every message has a format similar to that of a typical UNIX-style command-line program:

!report <region> human readable message

The bang (exclamation mark) at the beginning of the message is there so the system works with Maubot.

The human readable message can be anything. Such as "ICE Spotted at 500 E Fake St."

The region field is a single token (no spaces in it) which designates the state or territory that the report is for. This is a required field. This requirement and its granularity have undergone quite a bit of scrutiny. Its purpose is to allow for consumers of this information to filter by their general location without being too granular. The specific choice of state came down to most 50501 groups being organized around each state.

This is not case sensitive, but again it must be a single word/token. Just concatenate the words for state or territory names with multiple words. Like NorthDakota or puertoRico or COLORADO.

A more complete example might be:

!report arizona ICE seen at 500 N Central Ave in Phoenix`

Region List

In order to be extra careful, here's an exhaustive list of all reigons for The United States, listed in alphabetical order of their proper names.

Name	Reigon Name	Shortcode
Nationwide	global	ALL
Alabama	Alabama	AL
Alaska	Alaska	AK
American Samoa	AmericanSamoa	AS
Arizona	Arizona	AZ
Arkansas	Arkansas	AR
California	California	CA
Colorado	Colorado	CO
Connecticut	Connecticut	CT
Delaware	Delaware	DE
District of Columbia	WashingtonDC	DC
Florida	Florida	FL
Georgia	Georgia	GA
Guam	Guam	GU
Hawaii	Hawaii	HI
Idaho	Idaho	ID
Illinois	Illinois	IL
Indiana	Indiana	IN
Iowa	Iowa	IA
Kansas	Kansas	KS
Kentucky	Kentucky	KY
Louisiana	Louisiana	LA
Maine	Maine	ME
Maryland	Maryland	MD
Massachusetts	Massachusetts	MA
Michigan	Michigan	MI
Minnesota	Minnesota	MN
Mississippi	Mississippi	MS
Missouri	Missouri	MO
Montana	Montana	MT
Nebraska	Nebraska	NE
Nevada	Nevada	NV
New Hampshire	NewHampshire	NH
New Jersey	NewJersey	NJ
New Mexico	NewMexico	NM
New York	NewYork	NY
North Carolina	NorthCarolina	NC
North Dakota	NorthDakota	ND
Northern Mariana Islands	NorthernMarianaIslands	MP
Ohio	Ohio	OH
Oklahoma	Oklahoma	OK
Oregon	Oregon	OR
Pennsylvania	Pennsylvania	PA
Puerto Rico	PuertoRico	PR
Rhode Island	RhodeIsland	RI
South Carolina	SouthCarolina	SC
South Dakota	SouthDakota	SD
Tennessee	Tennessee	TN
Texas	Texas	TX
Utah	Utah	UT
U.S. Virgin Islands	USVirginIslands	VI
Vermont	Vermont	VT
Virginia	Virginia	VA
Washington	Washington	WA
West Virginia	WestVirginia	WV
Wisconsin	Wisconsin	WI
Wyoming	Wyoming	WY

And here's a convenient one-line list of the shortcodes:

all al ak as az ar ca co ct de dc fl ga gu hi id il in ia ks ky la me md ma mi mn ms mo mt ne nv nh nj nm ny nc nd mp oh ok or pa pr ri sc sd tn tx ut vi vt va wa wv wi wy

Input-Output & Authentication

This system is intended to be used in a Matrix room where different Matrix accounts are adding and consuming information. For simplicity, I will stick to the adders and consumers terminology. The adders can be bots or people, whereas the consumers are intended to be bots only.

Additionally, consumer bots would probably want to only pay attention to certain region designations. Additionally, consumer bots may only want to pay attention to certain users. All Maubot plugins that FiftyFiftyOne

As an example, a person could be the only adder, with two consumers—say a bot that forwards it to NTFY, and another one which forwards it to Signal.

When the person enters in !report arizona Some Report the two consumer bots should process it as follows:

1. Split the message into the region designator arizona, and the string after it.
2. If the bot is set to only pay attention to specific region designators, check that arizona is in that set. Silently ignore the message if it isn't.
3. If the bot is set to only pay attention to specific users for the region designator arizona, check if the sending user is in that set. Silently ignore the message if it isn't.
4. Run through its own code to process the message as it sees fit.
5. Optionally reply or react to the message saying that it was processed.

Maubot Specifics

Here's some sample code and configuration layout to make plugin development easier. You can copy paste these into your plugin, or use it as inspiration. FiftyFiftyOneArizona's plugins try to be as consistent as possible with its configuration options, and how the plugin behaves with that configuration.

Sample Configuration

Which users are allowed to send reports for which regions:

# Which regions the bot should pay attention to.
# Pass through empty list to allow all senders for a region.
# Add a list of MatrixIDs to whitelist senders for a region.
# Use __global__ to refer to all regions.
# Code looks to see if a sender is allowed for a specific region OR __global__.
allowed_regions:
    foo:
        - "@reports:fiftyfiftyonerarizona.org"
    bar:
        - "@reports:example.org"
        - "@reports:example.com"
    buzz:
    aaa:

Different configurations for each regions, and one which processes reports for all regions:

# Bot-specific configurations per region
# Code will assume no authentication if either username or password are missing.
# Use __global__ to specify an endpoint for all regions.
# Code will push to both __global__ and region-specific endpoint.
region_configs:
    arizona:
        # Where and how the plugin will send the data to
        server_url: "https://ntfy.sh"
        server_topic: "changeme"

        # If the plugin should use a username and password to send notifications
        username: "no"
        password: "way"

Defining which rooms a producer should send to for each region, and for all regions:

sendto:
    __global__:
        - "!example:example.com"
    arizona:
        - "!example:example.com"
        - "!example2:example.com"
    california:
        - "!example:example.com"

Helper Functions

Handling which users are allowed to send reports for which regions. Takes in region string and sender MatrixID as a string. Returns a boolean.

# Checks if a sender if allowed to send for a particular region
def validate_sender(self, region: str, sender: str):
	# Mautrix isn't documented, like at all, so I'm just gonna catch the
	# error because IDK how to see if a map is inside a map.
	try: allowed_list = self.config["allowed_regions"][region]
	except: return False
	
	if allowed_list is None: return True # Empty list
	if sender in allowed_list: return True
	
	# Sender not allowed in region config
	return False

Handling different configurations for each regions. Parses an event and its body, returns a list of configurations. List can be empty. Depends on validate_sender function above.

This function also handles the following edge cases

1. Sender allowed globally but __global__ is not defined in the region configs.
2. Sender allowed for a region, and both that region and __global__ is defined.
3. Sender is allowed either globally or for a region, but neither that region or __global__ are defined. An empty list will be returned.

# Does the necessary config checks for the given event
# Returns list of recursive configs to process
def validate_report(self, evt: MessageEvent, message: str):
	# Split command (minus !command_name) into tokens
	tokens = message.split()
	region = tokens[0].lower()
	
	# Each command must have a state/territory designation and a message
	if len(tokens) < 2: return []
	# And we must have self.config["region_configs"]
	try: trashvariable = self.config["region_configs"]
	except: return []
	
	configs = [] # To be returned
	
	allowed_globally = self.validate_sender("__global__", evt.sender)
	allowed_region = self.validate_sender(region, evt.sender)
	
	# If user is allowed globally and/or for a region,
	# the plugin should process both the region and __global__ configs
	if allowed_globally or allowed_region:
		for i in ["__global__", region]:
			try: configs.append(self.config["region_configs"][i])
			except: trashvariable = None
	
	return configs

Getting a set of rooms to send to, inclusive of __global__ and the region name.

def get_send_rooms(self, region_name):
	room_ids = set()
	
	if "sendto" in self.config: return room_ids
		for i in [region_name, "__global__"]:
			if i in self.config["sendto"]:
				for room_id in self.config["sendto"][i]:
					room_ids.add(room_id)
	
	return room_ids