Update build script

.woodpecker.yml

@@ -17,8 +17,8 @@ steps:
       - VITE_APP_NAME=Treadl
     commands:
       - cd web
-      - yarn install
+      - npm install
-      - yarn build
+      - npx vite build
 
   buildapi:
     group: build

README.md

@@ -2,19 +2,54 @@
 
 This is a monorepo containing the code for the web and mobile front-ends and web API for the Treadl platform.
 
-## Running and developing Treadl locally
 
-To run Treadl locally, we recommend taking the following steps:
+## Running Treadl locally in development mode
+
+To run Treadl locally, first ensure you have the needed software installed:
+
+- Python (3.12 is currently used)
+- Node.js (20.x is currently used)
+- Docker (we use this for the Mongo database)
+    - It can be installed via the Docker website or your package manager
+    - Ensure the Docker service is running
+- [Taskfile](https://taskfile.dev) (convenience tool for running tasks)
+    - This can be installed using `brew install go-task`
+
+To begin, clone this repository to your computer:
+
+```bash
+git clone https://git.wilw.dev/wilw/treadl.git
+```
+
+Next, initialise the project by installing dependencies and creating an environment file for the API:
+
+```bash
+task init
+```
+
+This generates a 'envfile' in your 'api' directory. You can edit this as needed (though the defaults should allow you to at least launch the app). Note: if you run this command again then any changes you made to your `envfile` will be overwritten.
+
+Finally, you can start the API and web UI by running:
+
+```bash
+task
+```
+
+Note: this command also starts the MongoDB database on port 27017. If the DB is already running, you'll see errors reported, but the API and web will still be launched.
+
+You can now navigate to [http://localhost:8002](http://localhost:8002) to start using the app.
+
+If you pull updates from the repository in the future (e.g. with `git pulll`) you may need to ensure your dependencies are up-to-date before starting the app again. This can be done with:
+
+```bash
+task install-deps
+```
 
-1. Check out this repository locally.
-1. Follow the instructions in the `api/` directory to launch a MongoDB instance and to run the Treadl API.
-1. Follow the instructions in the `web/` directory to install the local dependencies and run the web UI.
 
 ## Deploying your own version of Treadl
 
 If you'd like to launch your own version of Treadl in a web production environment, follow the steps below. These instructions set-up a basic version of Treadl, and you may want or need to take additional steps for more advanced options.
 
-We recommend forking this repository. That way you can make adjustments to the code to suit your needs, and pull in upstream updates as we continue to develop them.
 
 ### 1. Launch a MongoDB cluster/instance
 
@@ -35,6 +70,7 @@ Either way, once launched, make a note of the cluster/instance's:
 * URI: The database's URI, probably in a format like `mongodb+srv://USERNAME:PASSWORD@host.com/AUTHDATABASE?retryWrites=true&w=majority`
 * Database: The name of the database, within your cluster/instance, where you want Treadl to store the data.
 
+
 ### 2. Provision an S3-compatible bucket
 
 Treadl uses S3-compatible object storage for storing assets (e.g. uploaded files). You should create and configure a bucket for Treadl to use.
@@ -58,6 +94,7 @@ Once you have a bucket, generate some access keys for the bucket that will enabl
 
 _Note: assets in your bucket should be public. Treadl does not currently used signed requests to access uploaded files._
 
+
 ### 3. Provision the API
 
 The best way to run the web API is to do so via Docker. A `Dockerfile` is provided in the `api/` directory.
@@ -79,6 +116,7 @@ $ docker run --env-file envfile -d treadl-api
 
 _Note: a reverse proxy (such as Nginx or Traefik) should be running on your server to proxy traffic through to port 8000 on your running Treadl API container._
 
+
 ### 4. Host the front-end
 
 The front-end is formed from static files that can be simply served from a webserver, from a CDN-fronted object store, or anything else.
@@ -88,24 +126,13 @@ Before building or hosting the front-end, please copy the `.env.development` fil
 * Include the URL of the web API you deployed earlier in the relevant field.
 * Include a contact email address.
 
-**Vercel**
+Then, simply build the app and then deploy the resulting `dist/` directory to a server or storage of your choice:
-
-We use [Vercel](https://vercel.com) to host the web UI. Once you have an account to which you are logged-in to locally, the front-end can be deployed by simply running:
 
 ```shell
-$ vercel --prod
+$ npx vite build
+$ s3cmd cp dist/* s3://my-treadl-ui # Example
 ```
 
-_Note: You will need to configure Vercel to use your own domain, and set-up a project, etc. first._
-
-**Manual**
-
-Simply build the app and then deploy the resulting `build/` directory to a server or storage of your choice:
-
-```shell
-$ yarn build
-$ s3cmd cp build/ s3://my-treadl-ui # Example
-```
 
 ### 5. Optional extras
 
@@ -117,6 +144,7 @@ To use this feature, simply rebuild the app ensuring that an environment entry i
 
 _Note: If this is not set, Treadl will by default fetch the full size images straight from the S3 source._
 
+
 ## Contributions
 
 Contributions to the core project are certainly welcomed. Please [get in touch with the developer](https://wilw.dev) for an invitation to join this repository.

Taskfile.yml

@@ -0,0 +1,85 @@
+version: '3'
+
+vars:
+  VENV: "source .venv/bin/activate"
+
+tasks:
+  default:
+    desc: Run web bundler and API
+    deps:
+      - start-db
+      - run-api
+      - run-web
+
+  run-web:
+    desc: Run web frontend
+    dir: 'web'
+    cmds:
+      - echo "[Web] Starting React app..."
+      - npx vite --port 8002
+
+  run-api:
+    desc: Run API server
+    dir: 'api'
+    dotenv: ['envfile']
+    cmds:
+      - echo "[FLASK] Starting Flask app..."
+      - bash -c "{{.VENV}} && flask run --debug"
+
+  start-db:
+    desc: Start database
+    ignore_error: true
+    cmds:
+      - echo "[DB] Starting database..."
+      - docker run --rm -d --name mongo -v ~/.mongo:/data/db -p 27017:27017 mongo:6
+
+  init:
+    desc: Initialize project
+    cmds:
+      - task: install-deps
+      - cp api/envfile.template api/envfile
+
+  install-deps:
+    desc: Install all dependencies
+    deps:
+      - install-deps-web
+      - install-deps-api
+
+  install-deps-web:
+    desc: Install web dependencies
+    dir: 'web'
+    cmds:
+      - echo "[Web] Installing dependencies..."
+      - npm install
+
+  install-deps-api:
+    desc: Install API dependencies
+    dir: 'api'
+    cmds:
+      - echo "[FLASK] Installing dependencies..."
+      - cmd: virtualenv -p python3.12 {{.VENV}}
+        ignore_error: true
+      - bash -c "{{.VENV}} && pip install poetry"
+      - bash -c "{{.VENV}} && poetry install"
+
+  lint:
+    desc: Lint all
+    deps:
+      - lint-web
+      - lint-api
+
+  lint-web:
+    desc: Lint web frontend
+    dir: 'web'
+    cmds:
+      - echo "[Web] Linting React app..."
+      - npx standard --fix
+
+  lint-api:
+    desc: Lint API server
+    dir: 'api'
+    cmds:
+      - echo "[FLASK] Linting Flask app..."
+      - bash -c "{{.VENV}} && ruff format ."
+      - bash -c "{{.VENV}} && ruff check --fix ."
+

api/README.md

@@ -1,67 +1,3 @@
 # Treadl web API
 
 This directory contains the code for the back-end Treadl API.
-
-## Run locally
-
-To run this web API locally, follow the steps below.
-
-### 1. Run a local MongoDB instance
-
-Install MongoDB for your operating system and then launch a local version in the background. For example:
-
-```shell
-$ mongod --fork --dbpath=/path/to/.mongo --logpath /dev/null
-```
-
-(Remember to restart the database upon system restart or if the instance stops for another reason.)
-
-### 2. Create and activate a virtual environment
-
-Install and activate the environment using `virtualenv`:
-
-```shell
-$ virtualenv -p python3 .venv # You only need to run this the first time
-$ source .venv/bin/activate
-```
-
-### 3. Install dependencies
-
-We use Poetry to manage dependencies. If you don't have this yet, please refer to [the Poetry documentation](https://python-poetry.org) to install it. Once done, install the dependencies (ensuring you have `source`d your virtualenv first):
-
-```shell
-$ poetry install
-```
-
-### 4. Create an `envfile`
-
-Copy the template file into a new `envfile`:
-
-```shell
-$ cp envfile.template envfile
-```
-
-If you need to, make any changes to your new `envfile`. Note that changes are probably not required if you are running this locally. When happy, you can `source` this file too:
-
-```shell
-$ source envfile
-```
-
-### 5. Run the API
-
-Ensure that both the virtualenv and `envfile` have been loaded into the environment:
-
-```shell
-$ source .venv/bin/activate
-$ source envfile
-```
-
-Now you can run the API:
-
-```shell
-$ flask run
-```
-
-The API will now be available on port 2001.
-
-Remember that you will need a local instance of [MongoDB](https://www.mongodb.com) running for the API to connect to.

api/api/projects.py

@@ -1,7 +1,8 @@
 import datetime
 import re
+import os
 from bson.objectid import ObjectId
-from util import database, wif, util
+from util import database, wif, util, mail
 from api import uploads, objects
 
 default_pattern = {
@@ -325,7 +326,16 @@ def create_object(user, username, path, data):
                 if pattern:
                     obj["name"] = pattern["name"]
                     obj["pattern"] = pattern
-            except Exception:
+            except Exception as e:
+                mail.send(
+                    {
+                        "to": os.environ.get("ADMIN_EMAIL"),
+                        "subject": "Error loading WIF file",
+                        "text": "A WIF file failed to parse with error: {}. The contents are below:\n\n{}".format(
+                            e, data["wif"]
+                        ),
+                    }
+                )
                 raise util.errors.BadRequest(
                     "Unable to load WIF file. It is either invalid or in a format we cannot understand."
                 )

api/lint.sh

@@ -1,4 +0,0 @@
-#!/bin/bash
-
-ruff format .
-ruff check --fix .

api/pyproject.toml

@@ -1,6 +1,7 @@
 [tool.poetry]
 name = "api"
 version = "0.1.0"
+package-mode = false
 description = "Treadl API"
 authors = ["Will <will@treadl.com>"]
 

api/util/wif.py

@@ -152,7 +152,9 @@ def dumps(obj):
 
 
 def loads(wif_file):
-    config = configparser.ConfigParser(allow_no_value=True, strict=False)
+    config = configparser.ConfigParser(
+        allow_no_value=True, strict=False, inline_comment_prefixes=("#", ";")
+    )
     config.read_string(wif_file.lower())
     DEFAULT_TITLE = "Untitled Pattern"
     draft = {}

web/README.md

@@ -1,23 +1,3 @@
 # Treadl web UI
 
 This directory contains the code for the Treadl web front-end.
-
-## Run locally
-
-Firstly, please ensure a local version of the Treadl API is up and running by following the instructions in the `api/` directory.
-
-Then, to run the Treadl web UI locally, follow these steps;
-
-Install dependencies:
-
-```shell
-$ yarn
-```
-
-Run the app:
-
-```shell
-$ yarn start
-```
-
-Note that the development version of the front-end expects to be able to connect to the local API on port 2001.

web/package.json

@@ -29,11 +29,6 @@
     "styled-components": "^6.1.13",
     "use-debounce": "^10.0.3"
   },
-  "scripts": {
-    "start": "vite --port 8002",
-    "build": "vite build",
-    "lint": "standard --fix"
-  },
   "browserslist": [
     ">0.2%",
     "not dead",