Starting the web front-end

Oh boy. I've put this off long enough. Time to start the web front end.

Task number one - choose a web front-end framework. In previous projects, I've just stuck to writing plain Javascript, and I have avoided frameworks and big libraries as much as possible. But I don't think that's sustainable, and I really do need to learn this modern Javascript world. Candidates include React - the big fish in the pond - Angular, Vue and Svelte. There are dozens of other alternatives, of course.

I'm discounting React immediately, I think it's ugly, weird and created by evil Facebook. Once I would have chosen Angular - it's what we use at work - but I think it's pretty hard to read, and created by evil Google. I've briefly dabbled with Vue, but I've heard good things about Svelte so I'm going to try that out. But Svelte or Sveltekit - the documentation doesn't make it very clear which of the two best suits my needs. I know that I absolutely do not want a NodeJS server running in the background. I believe I can configure Sveltekit to generate "static" content, so I'll experiment with that.

It's a massive learning curve though - I'm not familiar with most of the terminology these frameworks use, I don't know Typescript, I barely know Javascript at times. It's hard for me to even google a sensible question as I don't know what I'm asking for.

Structure concerns

My project structure file has needed some tweaks (simplifications, really). But I have a more serious concern - the structure file is updated when a markdown file is uploaded to the source bucket. When multiple files are uploaded at once, there's a bit of a race condition and now the structure file becomes inconsistent, and usually missing some of the entries. I will need to change my approach to this, perhaps by adding a new SQS queue updateStructure (tough I am not sure if lambda will read all the queue items one-by-one, or try to run processes in parallel to consume the queue as quickly as possible.)

I definitely need to write an API to force a complete rebuild of the structure file too. Right now, to fix it I have to delete the file manually, then upload the markdown files one-by-one.

I am also increasingly worried that the lambda startup time is too great - fetching the structure file can take a long time, nearly a minute it seems.

Structure concerns update

I've decided not to automatically generate the structure file whenever a markdown file is uploaded. Instead, I've created a new API route which regenerates the entire structure json file. It seems to be working well. I'd also like an API route to update the structure file for a given source (patching in, really), and also one to delete a source, so I don't have to rebuild it all by scanning the source bucket each time. But that's a task for another day.