About

About

archive.wakarimasen.co is an archiving tech stack designed to explore more efficient and/or cheaper methods to run archives while having a straightforward and east upgrade path. This instance serves as sort of a stress designed to see how things can scale. This project is informally associated with Bibliotheca Anonoma.

Tech Stack

Software

The current software stack that powers this board is as follows and is open source.

  • Torako - This is a scraper which is designed to be compatible with Asagi. It goes a bit further in providing some options that optimize database load, as well as uploading images to cloud endpoints and pushing posts to a postgres search index. (Configuration)
  • FoolFuuka - This boards runs a fork of the FoolFuuka software; changes were made to the Search module to have it support Postgres rather than Sphinx. This will eventually be replaced by Ayase
  • MySQL - All posts are served from MySQL 8, using an Asagi compatible schema,
  • Postgresql - A separate posts index was created taking advantage of Postgres' Full Text search capabilities.
  • Backblaze - Media is stored in Backblaze for cheap storage
  • Cloudflare - Cloudflare is used to cheaply serve content

Hardware

Currently this board is hosted with a cloud provider

VPS 1: Torako

VPS 2: Fuuka

The remaining costs are to Backblaze, which is primarily storage @ $5/TB/mo

Archival Status

This archive fully went live on September 13, 2020 and should have everything from that date. Past posts are going to be imported into the database to study how data size affects performance, depending on how fast that data can be loaded and what data I can get my hands on.

DCMA / Abuse

See this page

Contact