Quick notes

This a special note that list various tips that are too short for a single “light note”, since I only list two light notes per page.

Firefox

How to import ‘recent’ Seamonkey and Waterfox profiles in recent Firefox releases

Locate the folder of your Seamonkey or Waterfox profile.
On Linux systems, it’s in ~/.mozilla/seamonkey and ~/.waterfox respectively.
On Windows, a random guess would be %APPDATA%\Mozilla\Seamonkey\Profiles or %APPDATA%\Waterfox\Profiles.

Check that keys4.db and logins.json are present. If only keys3.db is present, your installation is too old and won’t be imported correctly.

If these files are present, copy the profile folder into ~/.mozilla/firefox (or %APPDATA%\Mozilla\Firefox\Profiles).
In the same folder, you should see a profiles.ini file. Back it up and edit it.

This should look like this :

[Install4F96D1932A9F858E]
Default=rp4qlak7.default-release
Locked=1

[Profile1]
Name=default-release
IsRelative=1
Path=rp4qlak7.default-release
Default=1

[Profile0]
Name=default
IsRelative=1
Path=9ruyutsw.default

[General]
StartWithLastProfile=1
Version=2

The ‘Path’ will be different on your installation.

Add a [ProfileN] section where N is the next Profile number.

For example, if :

  • the profile folder you just copied is named abcdef12345.default
  • you have [Profile0] and [Profile1] sections in your profiles.ini file

Then add the following

[Profile2]
Name=toimport
IsRelative=1
Path=abcdef12345.default

The Name= is not very important. You should just remember it as you’ll have to choose it during the migration wizard.

Once done, close every window of Firefox and then start firefox with the --migration argument :

firefox --migration

On Windows, you’ll have to open a Powershell in the installation folder of Firefox, located in Program Files.

This should open a ‘Migration windows’. From this point :

  • Select Firefox
  • Click on Next >
  • Select toimport
  • Click on Next >
  • Then click on Finish

toimport is the name set in the profiles.ini just before.

Firefox should open the main page by now.
Check the “Logins and passwords” to ensure that, at least, your passwords were imported successfully.
Then check your bookmarks.

If the name toimport didn’t appear, check that you didn’t mess up the profiles.ini configuration. If that’s not the case, maybe your profile is really too old for a direct Firefox import.

Docker

standard_init_linux.go:207: exec user process caused "exec format error"

Either :

View the content of a running container, or post-mortem

docker export containerid -o /path/to/file.tar.gz

/path/to/file.tar.gz will be a snapshot of the / directory of your container.

No network during builds, while using custom networks

One solution would be to use the ‘host’ network just for the build, by adding a network: host node below your build node.

Complete example :

version: '3.4'

services:
  matrix:
    image: myy/synapse:latest
    build:
      context: ./build-synapse
      network: host
    networks:
      myynet:
        ipv6_address: fc00::105

networks:
  myynet:
    external: true

In this example, the Dockerfile path is ./build-synapse/Dockerfile
Also, the network myynet is not used during the build

Let’s encrypt (letsencrypt)

requests.exceptions.SSLError: HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443)

The complete error could be something like :

Max retries exceeded with url: /directory (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

Check your /etc/hosts if there’s a acme-v02.api.letsencrypt.org entry.

Then compare your result of :

$ nslookup
> acme-v02.api.letsencrypt.org

Between your current machine and another machine.
If the results are differents, you might to try using the same DNS server as your other machine, by :

  1. Modifying /etc/resolv.conf for quick-testing and adding a nameserver W.X.Y.Z entry at the top.
  2. Re-executing your certbot script.

If that works, configure your network to use the same DNS server, through the standard configuration files/utilities, since /etc/resolv.conf generally don’t persist after a reboot (or a DHCP lease renewal).

If that doesn’t work, try to use acme-v01.api.letsencrypt.org instead, by passing --server https://acme-v01.api.letsencrypt.org/directory to certbot.

Also make sure that your firewall isn’t blocking HTTPS egress connections, by checking up on https://google.com :

curl -L https://google.com

Systemd

I can connect to SSH but the port is wrong and the ssh.service is disabled ?

Turns out that some distributions install a ssh.socket in /etc/systemd/system/sockets.target.wants that, not only ignores your port configuration in /etc/ssh/sshd_config but also block ssh.service from running !

I’d like to say :
Just rm /etc/systemd/system/sockets.target.wants/ssh.socket
BUT ! It turns out that I got burned by this issue after a standard Debian unstable system update and a simple reboot… which means that you can easily be burned by this after a system update.

So… one alternative is to create a custom SSH service unit in /etc/systemd/system/ with a name like donttouchmyssh.service.
The content of /etc/systemd/system/donttouchmyssh.service like this :

[Unit]
Description=OpenBSD Secure Shell server
Documentation=man:sshd(8) man:sshd_config(5)
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run

[Service]
EnvironmentFile=-/etc/default/ssh
ExecStartPre=/usr/sbin/sshd -t
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/usr/sbin/sshd -t
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartPreventExitStatus=255
Type=notify
RuntimeDirectory=sshd
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target

The unit is based on /lib/systemd/system/ssh.service, but ours don’t advertise itself as ssh.service or sshd.service, meaning that ssh.socket won’t block it.

Test it by running :

systemctl start donttouchmyssh

And then try to connect on the SSH port you configured in the server /etc/ssh/sshd_config file.

If you can connect correctly, set this service to execute on startup

systemctl enable donttouchmyssh

Reboot and retry to connect on the configured port.

If you can, then you can use your firewall to block port 22 connections !
Block these ports from a machine, or an interface, which you can access if your custom service fails to start after another system update.

If you cannot, check for errors using journalctl -u donttouchmyssh and systemctl status donttouchmyssh (with sudo if required).

Iptables

Save / Restore

If you don’t have a iptables rules save/restore service, here’s a quick one for IPv4 and IPv6 rules.

In these services, rules will be loaded and saved from /etc/iptables/v4-rules and /etc/iptables/v6-rules, so prepare the environment before like this :

mkdir /etc/iptables
iptables-save > /etc/iptables/v4-rules
ip6tables-save > /etc/iptables/v6-rules

After that, create two services for loading and restoring IPv4 and IPv6 rules. The services must be saved in /etc/systemd/system.
You can name them iptables.service and ip6tables.service for example, but the name is up to you.

/etc/systemd/system/iptables.service

[Unit]
After=network.service
Description=Iptables IPv4 rules save/load service

[Service]
Type=oneshot
ExecStart=/bin/sh -c "iptables-restore < /etc/iptables/v4-rules"
ExceStop=/bin/sh -c "iptables-save > /etc/iptables/v4-rules"
RemainAfterExit=yes

[Install]
WantedBy=default.target

/etc/systemd/system/ip6tables.service

[Unit]
After=network.service
Description=Iptables IPv6 rules save/load service

[Service]
Type=oneshot
ExecStart=/bin/sh -c "ip6tables-restore < /etc/iptables/v6-rules"
ExceStop=/bin/sh -c "ip6tables-save > /etc/iptables/v6-rules"
RemainAfterExit=yes

[Install]
WantedBy=default.target

Test the services using :

systemctl start iptables && systemctl start ip6tables

Then check your firewall rules, using iptables -L, iptables -t nat -L, ip6tables -L and ip6tables -t nat -L.

If the rules appear ok, enable the script on startup and reboot

systemctl enable iptables
systemctl enable ip6tables
reboot

On reboot check the rules again, just to be sure.

If the rules aren’t good

If the rules are not good, correct them and then save them again in /etc/iptables/v4-rules and /etc/iptables/v6-rules using iptables-save and ip6tables-save respectively, then restart the iptables and ip6tables services like this :

systemctl restart iptables
systemctl restart ip6tables

Check the content of the files in /etc/iptables before and after the systemctl restart if the problem persist, in order to understand what’s going on.

Why is it blocking ?

If you happen to wonder why some packets are blocked in INPUT or OUTPUT, one simple way to get some clues is to set up a catch-all LOG rule :

INPUT

iptables -A INPUT -j LOG --log-level 6

OUTPUT

iptables -A OUTPUT -j LOG --log-level 6

Other cases

You can also put your catch-all sentence in any rule-chain, to see if the rule-chain is traversed :

iptables -A YOUR_RULE_CHAIN -j LOG --log-level 6

If you’re afraid that some rules might send the traffic to other rule chains, you can setup the LOG rule as the first rule of your chain. This will break your rulechain temporarily though.

iptables -I YOUR_RULE_CHAIN -j LOG --log-level 6

To check the logs, you can use journalctl on SystemD systems :

journalctl -k

Or just check dmesg or /var/log/messages on other systems.

/var/log/messages requires a syslog daemon logging kernel messages to this filepath.

Matrix

Synapse

(config for tls_private_key_path): No such file or directory even though no_tls: True is added

So if you server cannot start, with the following error :

matrix_1  | 2019-10-24 20:27:50,458 - twisted - 171 - ERROR -  - Traceback (most recent call last):
matrix_1  | 2019-10-24 20:27:50,459 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/app/_base.py", line 263, in start
matrix_1  | 2019-10-24 20:27:50,459 - twisted - 171 - ERROR -  -     refresh_certificate(hs)
matrix_1  | 2019-10-24 20:27:50,460 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/app/_base.py", line 212, in refresh_certificate
matrix_1  | 2019-10-24 20:27:50,461 - twisted - 171 - ERROR -  -     hs.config.read_certificate_from_disk(require_cert_and_key=True)
matrix_1  | 2019-10-24 20:27:50,462 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/config/tls.py", line 221, in read_certificate_from_disk
matrix_1  | 2019-10-24 20:27:50,462 - twisted - 171 - ERROR -  -     self.tls_private_key = self.read_tls_private_key()
matrix_1  | 2019-10-24 20:27:50,463 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/config/tls.py", line 487, in read_tls_private_key
matrix_1  | 2019-10-24 20:27:50,465 - twisted - 171 - ERROR -  -     private_key_pem = self.read_file(private_key_path, "tls_private_key_path")
matrix_1  | 2019-10-24 20:27:50,467 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/config/_base.py", line 135, in read_file
matrix_1  | 2019-10-24 20:27:50,468 - twisted - 171 - ERROR -  -     cls.check_file(file_path, config_name)
matrix_1  | 2019-10-24 20:27:50,469 - twisted - 171 - ERROR -  -   File "/usr/local/lib/python3.7/site-packages/synapse/config/_base.py", line 117, in check_file
matrix_1  | 2019-10-24 20:27:50,469 - twisted - 171 - ERROR -  -     % (file_path, config_name, e.strerror)
matrix_1  | 2019-10-24 20:27:50,470 - twisted - 171 - ERROR -  - synapse.config._base.ConfigError: Error accessing file '/data/matrix.miouyouyou.fr.tls.key' (config for tls_private_key_path): No such file or directory

Even though you seem to have enable no_tls: true in your /data/homeserver.yaml, check the two following things :

  1. The directory you are actually mounting has a file named homeserver.yaml.
    While this seems dumb, you have to understand that :
  2. If you setup the environment variable SYNAPSE_SERVER_NAME, this will generate a temporary configuration file that will overshadow your current configuration file.
    I.e. If you set SYNAPSE_SERVER_NAME, the server won’t look for your /data/homeserver.yaml but use an auto-generated one instead !.

Migrate your SQLite DB to a PostgreSQL DB using Docker

First, ensure that your synapse Docker /data is file-system accessible volume, or ensure at least that you can access this directory to create the new configuration file, and do a database copy.

Here, I’ll assume that you’re in a directory where the sub-directory data is mounted as /data in your Synapse Docker instance.

Quick and dirty

  • Create a volume for the PostgreSQL data

    docker volume create synapse_db_data

If you know what you’re doing and want to use a host directory for the database files, just replace following synapse_db_data references with /absolute/path/to/your/host/directory, like this : -v /absolute/path/to/your/host/directory:/var/lib/postgresql/data

  • Create a temporary environment variable for PostgreSQL configuration through Docker

    cat <<EOF > postgres.env
    POSTGRES_USER=your_db_user
    # If you actually use Y0urP4ssw0rd as a password, you're an idiot
    POSTGRES_PASSWORD=Y0urP4ssw0rd
    POSTGRES_DB=your_db_name
    EOF
  • Fire the PostgreSQL server

    docker run --rm --env-file docker.env -v synapse_db_data:/var/lib/postgresql/data --name synapsedb -d postgres

If the instance doesn’t start, remove --rm, then type docker logs synapsedb to view the logs, and then docker container rm synapsedb to get rid of the stopped container else you won’t be able to recreate it

  • Copy and modify your Synapse server homeserver.yaml configuration to use PostgreSQL

    cp data/homeserver.yaml data/homeserver-pgsql.yaml

Then modify the section that look like this :

## Database ##

database:
  # The database engine name
  name: "sqlite3"
  # Arguments to pass to the engine
  args:
    # Path to the database
    database: "/data/homeserver.db"

and make it look this :

database:
  # The database engine name
  name: "psycopg2"
  # Arguments to pass to the engine
  args:
    user: your_db_user
    password: Y0urP4ssw0rd
    database: your_db_name
    host: synapsedb
  • Stop your current Synapse server

    docker stop your_synapse_container_tag_or_numeric_id
  • Copy your SQLite database just in case

    cp data/homeserver.db{,.bak}
  • Run the migration script with a new synapse instance

Replace myy/synapse-image:latest-intel by the reference of your synapse image.

docker run --name synapse  -v $PWD/data:/data --entrypoint synapse_port_db -d myy/synapse-image:latest-intel --sqlite-database /data/homeserver.db --postgres-config /data/homeserver-postgres.yaml

If you haven’t build your own synapse-image, either use the official one, or build one using my guide.

  • Follow the migration

    docker logs -f synapse
  • Shutdown your PostgreSQL instance

    docker stop synapsedb
  • Launch the whole thing with a docker-compose.yml

Replace myy/synapse-image:latest-intel by the reference of your synapse image.

( Add docker secrets commands )

version: '3'

services:
  matrix:
    image: myy/synapse:latest-intel
    build:
      context: ./build-synapse
      network: host
    volumes:
      - "./data:/data"
    networks:
      myynet:
        ipv6_address: fc00::110
  postgresql:
    image: postgres
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=your_db_user
      - POSTGRES_PASSWORD=Y0urP4ssw0rd
      - POSTGRES_DB=your_db_name

networks:
  myynet:
    external: true

Notes

Use a specific network for these containers

If you want to use a specific network for the generated containers, add this to the commands :

--net networkname --ip your.ip.v.4 --ip6 your::ipv6

Example with the PostgreSQL instance :

docker run --net networkname --ip your.ip.v.4 --ip6 your::ipv6 --rm --env-file docker.env -v synapse_db_data:/var/lib/postgresql/data --name synapsedb -d postgres

If you want to use a specific network in the docker-compose.yml, you’ll have to add the following nodes under each service :

networks:
  yournetworkname:
    ipv4_address: your.ip.v.4
    ipv6_adderss: your::ipv6

And add the following at end of the file :

networks:
  yournetworkname:
    external: true

Example :

version: '3'

services:
  matrix:
    image: myy/synapse:latest-intel
    build:
      context: ./build-synapse
      network: host
    volumes:
      - "./data:/data"
    networks:
      yournetworkname:
        ipv4_address: your.ip.v.4
        ipv6_address: your::ipv6
  postgresql:
    image: postgres
    volumes:
      - synapse_pg_data:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER: coincoin
      - POSTGRES_PASSWORD:

networks:
  yournetworkname:
    external: true

Check Docker compose official documentation for more details.