- latest posts + latest posts
++ all the posts, in reverse chronological. there is an atom feed above. +
-
diff --git a/_site/blog/tag/bootstrap/index.html b/_site/blog/tag/bootstrap/index.html
index e69de29..63de065 100644
--- a/_site/blog/tag/bootstrap/index.html
+++ b/_site/blog/tag/bootstrap/index.html
@@ -0,0 +1,142 @@
+
+
+
+
+
+
+
+
+
+
-
+
A measure of how easily thoughts can be translated to action
++ 10 Mar 2023 | + thoughts ++ Recently, I’ve been using multiple different languages and frameworks (both at work and personally) and I’ve been thinking about why I prefer some over others. Ultimately, I think it comes... +
+
+
+
+ -
+
Musings on the power of cinema
++ 14 Jan 2023 | + thoughts ++ I’ve realised that going to the cinema makes me sad; a quiet, contemplative melancholy. Ennui. No matter the film, something about the cinema experience is meditative for me. I always... +
+
+
+
+ -
+
Links aren't performances
++ 29 Apr 2023 | + links ++ “For starters, it finds that publishing a link can be a “performance” of a copyrighted work, which meant that when Linkletter linked to the world-viewable Youtube files that Proctorio had... +
+
+
+
+ -
+
Running a Python script periodically in a Docker container using cron
++ 19 Apr 2023 | + til ++ Recently, my partner gave a great idea for utilising my old Kindle: generate a “newspaper” each morning from a bunch of RSS feeds, and email it to the Kindle using... +
+
+
+
+ -
+
A measure of how easily thoughts can be translated to action
++ 10 Mar 2023 | + thoughts ++ Recently, I’ve been using multiple different languages and frameworks (both at work and personally) and I’ve been thinking about why I prefer some over others. Ultimately, I think it comes... +
+
+
+
+ -
+
Running a Python script periodically in a Docker container using cron
++ 19 Apr 2023 | + til ++ Recently, my partner gave a great idea for utilising my old Kindle: generate a “newspaper” each morning from a bunch of RSS feeds, and email it to the Kindle using... +
+
+
+
+
+
+ -
+
Fixing php errors in a Nextcloud docker-compose configuration
++ 01 Mar 2023 | + til ++ I was trying to rescan the files in my Nextcloud server (running on Raspberry Pi 4 with DietPi), and kept running into some weird php errors. First error - “could... +
+
+
+
+ -
+
Running a Python script periodically in a Docker container using cron
++ 19 Apr 2023 | + til ++ Recently, my partner gave a great idea for utilising my old Kindle: generate a “newspaper” each morning from a bunch of RSS feeds, and email it to the Kindle using... +
+
+
+
+
+
+ -
+
Fixing php errors in a Nextcloud docker-compose configuration
++ 01 Mar 2023 | + til ++ I was trying to rescan the files in my Nextcloud server (running on Raspberry Pi 4 with DietPi), and kept running into some weird php errors. First error - “could... +
+
+
+
+ -
+
Musings on the power of cinema
++ 14 Jan 2023 | + thoughts ++ I’ve realised that going to the cinema makes me sad; a quiet, contemplative melancholy. Ennui. No matter the film, something about the cinema experience is meditative for me. I always... +
+
+
+
+ -
+
A measure of how easily thoughts can be translated to action
++ 10 Mar 2023 | + thoughts ++ Recently, I’ve been using multiple different languages and frameworks (both at work and personally) and I’ve been thinking about why I prefer some over others. Ultimately, I think it comes... +
+
+
+
+ -
+
A measure of how easily thoughts can be translated to action
++ 10 Mar 2023 | + thoughts ++ Recently, I’ve been using multiple different languages and frameworks (both at work and personally) and I’ve been thinking about why I prefer some over others. Ultimately, I think it comes... +
+
+
+
+ -
+
Musings on the power of cinema
++ 14 Jan 2023 | + thoughts ++ I’ve realised that going to the cinema makes me sad; a quiet, contemplative melancholy. Ennui. No matter the film, something about the cinema experience is meditative for me. I always... +
+
+
+
+ -
+
Running a Python script periodically in a Docker container using cron
++ 19 Apr 2023 | + til ++ Recently, my partner gave a great idea for utilising my old Kindle: generate a “newspaper” each morning from a bunch of RSS feeds, and email it to the Kindle using... +
+
+
+
+ -
+
Links aren't performances
++ 29 Apr 2023 | + links ++ “For starters, it finds that publishing a link can be a “performance” of a copyrighted work, which meant that when Linkletter linked to the world-viewable Youtube files that Proctorio had... +
+
+
+
+ -
+
Yuri Felsen on love
++ 05 Mar 2023 | + quotes ++ “Like everyone, I have my own, maybe obsessive futile, maybe in some way authentic, vision: all of a sudden, I will imagine the entire homogenous world as it is revealed... +
+
+
+
+ -
+
Fixing php errors in a Nextcloud docker-compose configuration
++ 01 Mar 2023 | + til ++ I was trying to rescan the files in my Nextcloud server (running on Raspberry Pi 4 with DietPi), and kept running into some weird php errors. First error - “could... +
+
+
+
+ -
+
Fixing php errors in a Nextcloud docker-compose configuration
++ 01 Mar 2023 | + til ++ I was trying to rescan the files in my Nextcloud server (running on Raspberry Pi 4 with DietPi), and kept running into some weird php errors. First error - “could... +
+
+
+
+ -
+
Links aren't performances
++ 29 Apr 2023 | + links ++ “For starters, it finds that publishing a link can be a “performance” of a copyrighted work, which meant that when Linkletter linked to the world-viewable Youtube files that Proctorio had... +
+
+
+
+ -
+
Running a Python script periodically in a Docker container using cron
++ 19 Apr 2023 | + til ++ Recently, my partner gave a great idea for utilising my old Kindle: generate a “newspaper” each morning from a bunch of RSS feeds, and email it to the Kindle using... +
+
+
+
+ -
+
A measure of how easily thoughts can be translated to action
++ 10 Mar 2023 | + thoughts ++ Recently, I’ve been using multiple different languages and frameworks (both at work and personally) and I’ve been thinking about why I prefer some over others. Ultimately, I think it comes... +
+
+
+
+ -
+
Yuri Felsen on love
++ 05 Mar 2023 | + quotes ++ “Like everyone, I have my own, maybe obsessive futile, maybe in some way authentic, vision: all of a sudden, I will imagine the entire homogenous world as it is revealed... +
+
+
+
+ - crontab.guru is a great resource for checking
cronsyntax
+ - Installing vim/nano in your Docker container to make the debugging stage easier. This is especially useful for changing your crontab to run much more frequently, or adding debugging messages etc., when the container is up. +
+ + ac + +
+
+ bootstrap +
++
-
+
+
+
+
+
+
+ + ac + +
++ cinema +
++
-
+
+
+
+
+
+
+ + ac + +
++ copyright +
++
-
+
+
+
+
+
+
+ + ac + +
++ cron +
++
-
+
+
+
+
+
+
+ + ac + +
++ css +
++
-
+
+
+
+
+
+
+ + ac + +
++ docker-compose +
++
-
+
+
+
+
+
+
+ + ac + +
++ docker +
++
-
+
+
+
+
+
+
+ + ac + +
++ film +
++
-
+
+
+
+
+
+
+ + ac + +
++ html +
++
-
+
+
+
+
+
+
+ + ac + +
++ jekyll +
++
-
+
+
+
+
+
+
+ + ac + +
++ journal +
++
-
+
+
+
+
+
+
+ + ac + +
++ kindle +
++
-
+
+
+
+
+
+
+ + ac + +
++ law +
++
-
+
+
+
+
+
+
+ + ac + +
++ love +
++
-
+
+
+
+
+
+
+ + ac + +
++ nextcloud +
++
-
+
+
+
+
+
+
+ + ac + +
++ php +
++
-
+
+
+
+
+
+
+ + ac + +
++ pluralistic +
++
-
+
+
+
+
+
+
+ + ac + +
++ python +
++
-
+
+
+
+
+
+
+ + ac + +
++ webdev +
++
-
+
+
+
+
+
+
+ + ac + +
++ yuri felsen +
++
-
+
+
+
+
+
+
This is a perverse, even surreal take on copyright. The court rejects Linkletter’s argument that even Youtube’s terms of service warned Proctorio that publishing world-viewable material on its site constituted permission for people to link to and watch that material.”
-Via: Pluralistic
]]>Via: Pluralistic
]]>I loved this idea, and thought it would be no problem to get a Python script up and running periodically on my Raspberry Pi home server using cron. However, I ran into various issues along the way (some of which were not so easy to resolve), so I’m collating all the configuration changes I made in the hopes that it will be useful to someone one day. You can find the full repo for this project here, and I have also included my Dockerfile, docker-compose.yml and crontab at the end of this TIL.
You can then add this to the top of your Python script, as follows:
#!/usr/bin/local/python3
3. Still missing dependecencies
+3. Still missing dependencies
Some modules will still run into errors even when the PYTHONPATH variable has been set. In particular, I ran into problems with reportlab and Pillow/PIL:
ImportError: cannot import name '_imaging' from 'PIL'
How easily can I translate what I am thinking about into a working setup?
@@ -118,11 +118,11 @@ PYTHONPATH=/usr/bin/local/python3These are usually a direct tradeoff, with ease typically preferred over options. This tends to frustrate me, as I want full control of all those little configurations.
-The solution to this, at least for me, are frameworks such as Bootstrap and Jekyll. They put a layer between the user and the options; essentially translating and simplying all the little configurations into something much easier to work with. Columns in Bootstrap make page layouts simple; markdown-to-html parsing in Jekyll makes text formatting a breeze. But, most importantly of all, these frameworks don’t sacrifice on the possibilities. They make it easier to apply complex setups, but if you want to go in and tweak absolutely everything to your liking, go right ahead. By providing the user with simpler ways of doing the same things as before, they manage to bridge the gap between Can I do everything I want to? and How easy is it?. And I think that is beautiful.
]]>The solution to this, at least for me, are frameworks such as Bootstrap and Jekyll. They put a layer between the user and the options; essentially translating and simplying all the little configurations into something much easier to work with. Columns in Bootstrap make page layouts simple; markdown-to-html parsing in Jekyll makes text formatting a breeze. But, most importantly of all, these frameworks don’t sacrifice on the possibilities. They make it easier to apply complex setups, but if you want to go in and tweak absolutely everything to your liking, go right ahead. By providing the user with simpler ways of doing the same things as before, they manage to bridge the gap between Can I do everything I want to? and How easy is it?. And I think that is beautiful.
]]>- Yuri Felsen (trans. Bryan Karetnyk)
-From ‘Deceit’, first published in 1930 and translated to English in 2022.
]]>php errors.
+From ‘Deceit’, first published in 1930 and translated to English in 2022.
]]>php errors.
First error - “could not find driver”
@@ -164,4 +164,4 @@ This led me to running this command:….aaaaaand, voila! The command runs, the files are scanned and everything is up to date.
-
I loved this idea, and thought it would be no problem to get a Python script up and running periodically on my Raspberry Pi home server using cron. However, I ran into various issues along the way (some of which were not so easy to resolve), so I’m collating all the configuration changes I made in the hopes that it will be useful to someone one day. You can find the full repo for this project here, and I have also included my Dockerfile, docker-compose.yml and crontab at the end of this TIL.
1. Double check the user
+ +A lot of problems with cron come down to user privileges. Each user has their own crontab, and then there is the system-wide root crontab. The first issue I ran into with creating a cron job inside a container was that Docker created the crontab as a non-root user. This issue presented itself to me when I tried to run the following command, to list the current cronjobs in the Docker container:
docker-compose exec container-name crontab -l
+This returned the following output:
+no crontab for root
+Now, it is not necessarily a problem to have non-root cron jobs, but just make absolutely certain that you are creating the jobs with the user you expect. For me, I wanted to run as root, so I added to following line to my docker-compose.yml:
user: root
+Now, the root user will be used when building your Docker image and the created crontab will be where you expect.
2. Missing dependencies
+When cron calls your Python script, you may run into issues with ModuleNotFoundError or ImportError, where Python cannot find your installed packages. This is because cron does not have access to your system environment variables, including the Python path. You can resolve most of these errors with imports by adding the PYTHONPATH environment variable to your crontab. This should be the path to your site-packages folder, something like this:
PYTHONPATH=/usr/bin/local/python3
+You may also need to add a shebang (#!) to your Python script to direct cron to the correct version. You can find the Python location with one of the following commands:
which python
+which py
+which python3
+NOTE: These commands must be performed in your Docker container when it is up and running. In docker-compose syntax this would be the following (with the name of your container instead of container-name):
docker-compose exec container-name which python3
+You can then add this to the top of your Python script, as follows:
+#!/usr/bin/local/python3
+3. Still missing dependencies
+Some modules will still run into errors even when the PYTHONPATH variable has been set. In particular, I ran into problems with reportlab and Pillow/PIL:
ImportError: cannot import name '_imaging' from 'PIL'
+This was solved by adding the system PATH to the crontab as well. The system path is included in the default crontab that is created when you first run crontab -e:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+Therefore, it is a good idea to include it if you are making a new crontab to make sure cron can find everything it needs to.
4. Check relative paths in Python
+By default, cron runs from the default root path. Therefore, both your call to Python in your crontab and the filepaths within Python should either be relative to root (i.e /main.py rather than main.py) or just use full paths instead.
5. “Failed to build wheel” and related errors
+This error is related to Python inside a Docker container rather than cron. However, someone might still find it useful. When you install your requirements.txt, you may encounter errors such as
legacy-install-failure
+error: command '/usr/bin/gcc' failed with exit code 1
+fatal error: Python.h: No such file or directory
+I was able to resolve these by adding python3-dev, wheel and Cmake to my requirements.txt. These are sometimes required when packages include other binaries or need to compile other code when installed.
6. Other useful tips
+-
+
I hope this helped you resolve some errors! I’ve included my Dockerfile, docker-compose.yml and crontab below if you want to set up a similar project or adjust your own files. The full repo is also available here.
+ +Dockerfile:
+FROM python:3
+
+COPY . .
+RUN python3.11 -m pip install --no-cache-dir -r requirements.txt
+
+RUN touch /var/log/cron.log
+
+RUN apt-get update \
+&& apt-get install cron -y
+
+RUN chmod +x main.py
+
+RUN crontab crontab
+
+CMD cron -f
+docker-compose.yml:
+version: "2.4"
+
+services:
+ watchman:
+ platform: "linux/arm64/v8"
+ image: watchman:latest
+ container_name: watchman
+ restart: always
+ user: root
+ build:
+ context: build
+ dockerfile: Dockerfile
+crontab:
+PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+PYTHONPATH=/usr/bin/local/python3
+15 7 * * * python3 /main.py >> /var/log/cron.log 2>&1
+
+php errors.
+
+First error - “could not find driver”
+ +I was trying using the following syntax to call occ and scan the files:
sudo -u www-data php /path/to/nextcloud/occ files:scan --all
+but I kept running into a PHP error. Specifically this error:
+ +Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: could not find driver in /path/to/nextcloud/lib/private/DB/Connection.php:139
+followed by a long, verbose stack trace.
+ +It took me a decent amount of time to diagnose the exact issue, but eventually I found this list of required PHP modules in the Nextcloud admin manual.
+ +Running php -m will print out the list of currently installed PHP modules. I noticed I was missing quite a few of the required modules, but the one that was causing my issue was the missing pdo_mysql module.
+This can be installed by running:
sudo apt-get install php7.4-mysql
+Note: This command will change based on your OS, PHP version and database type
+ +This resolved the error! However (as is always the case), this only meant I got a shiny new error instead:
+ +Second error - “Name or service not known”
+ +Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known in /path/to/nextcloud/lib/private/DB/Connection.php:139
+From first glance, this looks like something wrong in the DNS name resolution. This sent me a long way down the wrong path, changing a whole bunch of things in my docker-compose.yml file.
+Eventually however, after a long and perilous journey over the high seas of Nextcloud forums and StackOverflow, I found this example of running php occ in a docker-compose configuration.
+This led me to running this command:
docker-compose exec -u www-data nextcloud-app php occ files:scan --all
+Note: replace nextcloud-app with the name of your Nextcloud container. Also, this command must be run from the directory of your Nextcloud docker-compose.yml
+ +….aaaaaand, voila! The command runs, the files are scanned and everything is up to date.
+ +
+ til +
+photos↗ @@ -91,6 +98,10 @@ + + + +
about me
++ +
-Hi! My name is Andrew Conlin. I am a software engineer and photographer, currently based in Cambridge, England. +Hi! My name is Andrew Conlin. I am a software engineer and photographer, currently based in Cambridge, England.
My main hobbies are film photography and running. I love watching movies, especially in the cinema. I love post-punk, dreampop and experimental hip hop. I love drinking good coffee, good beer and good whisky. I am also a passionate self-hoster, and like to think that I'm at least somewhat privacy conscious. diff --git a/_site/til/index.html b/_site/til/index.html index 7538751..1c140a3 100644 --- a/_site/til/index.html +++ b/_site/til/index.html @@ -5,6 +5,7 @@ +
+ til +
+photos↗ @@ -91,6 +98,10 @@ + + + +
- latest tils + latest tils
++ my collection of tils (today-i-learned). i use this mainly to collect solutions to weird errors i have faced, and detail the little projects that i work on. this has its own atom feed (above) as they are typically very different from my other posts. all tils are blog posts but not all blog posts are tils. +
-
diff --git a/_site/til/nextcloud-php-errors/index.html b/_site/til/nextcloud-php-errors/index.html
index 9d8ed35..82ad9e9 100644
--- a/_site/til/nextcloud-php-errors/index.html
+++ b/_site/til/nextcloud-php-errors/index.html
@@ -5,6 +5,7 @@
+
+ til +
+photos↗ @@ -91,6 +98,10 @@ + + + +
Fixing php errors in a Nextcloud docker-compose configuration
++ +
diff --git a/_site/til/running-python-in-cron-in-docker/index.html b/_site/til/running-python-in-cron-in-docker/index.html index 146d78a..cfb4f28 100644 --- a/_site/til/running-python-in-cron-in-docker/index.html +++ b/_site/til/running-python-in-cron-in-docker/index.html @@ -5,6 +5,7 @@ ++ til +
+photos↗ @@ -91,6 +98,10 @@ + + + +
Running a Python script periodically in a Docker container using cron
++ +
@@ -174,7 +188,7 @@ which python3You can then add this to the top of your Python script, as follows:
#!/usr/bin/local/python3
3. Still missing dependecencies
+3. Still missing dependencies
Some modules will still run into errors even when the PYTHONPATH variable has been set. In particular, I ran into problems with reportlab and Pillow/PIL:
ImportError: cannot import name '_imaging' from 'PIL'
-Hi! My name is Andrew Conlin. I am a software engineer and photographer, currently based in Cambridge, England. +Hi! My name is Andrew Conlin. I am a software engineer and photographer, currently based in Cambridge, England.
My main hobbies are film photography and running. I love watching movies, especially in the cinema. I love post-punk, dreampop and experimental hip hop. I love drinking good coffee, good beer and good whisky. I am also a passionate self-hoster, and like to think that I'm at least somewhat privacy conscious. diff --git a/til.html b/til.html index ad689aa..bd29107 100644 --- a/til.html +++ b/til.html @@ -1,6 +1,7 @@ --- layout: default -title: latest tils +title: latest tils +subtitle: my collection of tils (today-i-learned). i use this mainly to collect solutions to weird errors i have faced, and detail the little projects that i work on. this has its own atom feed (above) as they are typically very different from my other posts. all tils are blog posts but not all blog posts are tils. permalink: /til/ categories: - til