Media Library Step 3: Coding the Backend (Part 2)
Now let's add in the rest of the routes; they're rather short. Then we'll present you with the entire code.
Here's a route that returns the IDs of only the thumbnails; the idea is that these could (and will) be used to build a page that generates a bunch of IMG html tags with the source set to the GET /image/:id route:
app.get('/thumbnails', async (req, res) => {
// todo: Consider building an index, as pulling back all the thumbnail data via query is a lot of unnecessary overhead
const thumbs = await client.queryEntities(
'type="thumbnail" && app="golem-images-0.1"'
)
res.send(
thumbs.map((item) => {
return item.entityKey
})
)
})
Here we use the queryEntities call, specifying the type as thumbnail and the app name. That way we'll get every instance of thumbnail available to us for this particular app.
Note: The queryEntities presently also returns the data, but we're not using that here; thus we loop through each one via the map function, returning only the entity's key.
Next we need a special route that given the thumbnail ID, return the "real" or "parent" image ID. The idea is that the user will be able to click on a thumbnail pic, and then open the real pic. (I'll be frank here; we debated whether to return the parent image itself, or only it's ID. We settled on ID, but the app could still function well, perhaps even more optimally, if we simply returned the parent image. You might experiment with returning the image and see how that works out.)
Here's the code:
app.get('/parent/:thumbid', async (req, res) => {
let id: Hex = prepend0x(req.params.thumbid)
// Get the metadata
const metadata = await client.getEntityMetaData(id as Hex)
if (metadata) {
for (let annot of metadata.stringAnnotations) {
if (annot.key == 'parent') {
// Not sure yet, let's just return the parent key for now and see how that works
res.send(annot.value)
return
}
}
// No parent key found
res.status(404)
res.send('not found')
return
} else {
res.status(404)
res.send('not found')
return
}
})
And finally, we have the required express server code:
app.listen(port, () => {
console.log(`Server is running at http://localhost:${port}`)
})
That's it for the backend! We'll provide you with the entire code at the end of this page.
Ready to run it? Make sure you have the script tags in your package.json, and type:
npm run build
to build it, and make sure there aren't any errors. Then to run it:
npm run start
Even without a front end yet, you can test out its built in static HTML page for a form that lets you upload images. And you can easily test out all the routes in Postman.
The full code
(You can also find the full code at https://github.com/frecklefacelabs/golembase-images).
import express from 'express'
import cors from 'cors'
import multer from 'multer'
import sharp from 'sharp'
import { inspect } from 'util'
import {
AccountData,
Annotation,
createClient,
GolemBaseCreate,
Hex,
Tagged,
} from 'golem-base-sdk'
import { readFileSync } from 'fs'
const app = express()
const port = 3000
const corsOptions = {
origin: 'http://localhost:4200',
}
app.use(cors(corsOptions))
app.use(express.json())
// Configure multer to handle file uploads in memory
// This means the file will be available as a Buffer on `req.file`
const storage = multer.memoryStorage()
const upload = multer({ storage: storage })
const keyBytes = readFileSync('./private.key')
const key: AccountData = new Tagged('privatekey', keyBytes)
export const client = await createClient(
1337,
key,
'http://localhost:8545',
'ws://localhost:8545'
)
app.get('/', (req, res) => {
res.send(`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Image Uploader</title>
<style>
body { font-family: sans-serif; max-width: 600px; margin: 2em auto; }
form { display: flex; flex-direction: column; gap: 1em; }
input, button { padding: 0.5em; }
</style>
</head>
<body>
<h1>Upload an Image</h1>
<form action="http://localhost:3000/upload" method="POST" enctype="multipart/form-data">
<div>
<label for="imageFile">Choose image:</label>
<input type="file" id="imageFile" name="imageFile" accept="image/*" required />
</div>
<div>
<label for="filename">Filename (if you want it different from original):</label>
<input type="text" id="filename" name="filename" />
</div>
<div>
<label for="tags">Tags (comma-separated):</label>
<input type="text" id="tags" name="tags" value="landscape, nature, sunset" required />
</div>
<div for="custom_key1">Optional Custom Tags (Key, Value)</div>
<div>
<input type="text" id="custom_key1" name="custom_key1" value="" />
<input type="text" id="custom_value1" name="custom_value1" value="" />
</div>
<div>
<input type="text" id="custom_key2" name="custom_key2" value="" />
<input type="text" id="custom_value2" name="custom_value2" value="" />
</div>
<div>
<input type="text" id="custom_key3" name="custom_key3" value="" />
<input type="text" id="custom_value3" name="custom_value3" value="" />
</div>
<button type="submit">Upload</button>
</form>
</body>
</html>
`)
})
const prepend0x = (id: string): Hex => {
// Prepend '0x' if it's missing
if (!id.startsWith('0x')) {
id = '0x' + id
}
return id as Hex
}
interface ImageResult {
id: string | null
image_data: Buffer
filename: string
mimetype: string
}
const getFullImage = async (id: Hex) => {
// For those not familiar with Partial, it's a great way to build up the object as we go
// without having to put a bunch of | null's at the end of each type in the Interface
// (because we don't want them to be null when we return the object.)
// Here's the ref: https://www.typescriptlang.org/docs/handbook/utility-types.html#partialtype
let result: Partial<ImageResult> = {
id: id,
mimetype: '',
filename: '',
}
// Grab the metadata
const metadata = await client.getEntityMetaData(id)
console.log(metadata)
// Grab the filename and mime type
let filename = 'image'
let partof = 1
for (let annot of metadata.stringAnnotations) {
if (annot.key == 'filename') {
filename = annot.value
} else if (annot.key == 'mime-type') {
result.mimetype = annot.value
}
}
for (let annot of metadata.numericAnnotations) {
if (annot.key == 'part-of') {
partof = annot.value
}
}
result.filename = filename
console.log(filename)
console.log(result.mimetype)
console.log(partof)
console.log('Fetching raw data...')
result.image_data = Buffer.from(await client.getStorageValue(id as Hex))
// See if there are more parts.
if (partof > 1) {
const chunks = [result.image_data]
// The query only gives us the payload and not the metadata, so we'll query them each individually
// (Note that we saved the values 1-based not 0-based, so the second has index 2 now)
for (let i = 2; i <= partof; i++) {
const chunk_info = await client.queryEntities(
`parent="${id}" && type="image_chunk" && app="golem-images-0.1" && part=${i}`
)
console.log(`CHUNKS ${i}:`)
console.log(chunk_info)
chunks.push(chunk_info[0].storageValue as Buffer)
}
console.log(`SENDING ${chunks.length} chunks`)
result.image_data = Buffer.concat(chunks)
}
return result as ImageResult
}
app.get('/image/:id', async (req, res) => {
let id: Hex = prepend0x(req.params.id)
let result: ImageResult = await getFullImage(id)
res.set('Content-Disposition', `inline; filename="${result.filename}"`)
res.type(result.mimetype)
res.send(result.image_data)
})
app.post('/upload', upload.single('imageFile'), async (req, res) => {
try {
let entity_key = ''
// --- 1. VALIDATE THE INPUT ---
// Check if a file was uploaded
if (!req.file) {
return res.status(400).send('No image file was uploaded.')
}
console.log('Filename:')
console.log(req.body.filename || req.file.originalname)
// Check for the tags field
console.log(req.body)
const { tags } = req.body
if (!tags || typeof tags !== 'string') {
return res.status(400).send('Tags string is required.')
}
console.log(`Received upload with tags: "${tags}"`)
let stringAnnotations = []
let numericAnnotations = []
// Add each tag as an annotation.
const tag_list = tags
.split(',') // split by commas
.map((tag) => tag.trim()) // remove leading/trailing space
.filter((tag) => tag.length > 0) // remove empty strings resulting from multiple commas
for (let tag of tag_list) {
stringAnnotations.push(new Annotation('tag', tag))
}
for (let i = 1; i <= 3; i++) {
const key = req.body[`custom_key${i}`]
const value = req.body[`custom_value${i}`]
if (key && value) {
console.log(`Found custom key/value ${i}:`)
console.log(key, value)
if (typeof value === 'number' && !isNaN(value)) {
numericAnnotations.push(new Annotation(key, value))
} else {
stringAnnotations.push(new Annotation(key, String(value)))
}
}
}
// --- 2. GET THE ORIGINAL IMAGE DATA ---
// The original image is already in memory as a Buffer
const originalImageBuffer = req.file.buffer
console.log(`Original image size: ${originalImageBuffer.length} bytes`)
// --- 3. RESIZE THE IMAGE USING SHARP ---
// sharp takes the buffer, resizes it, and outputs a new buffer
console.log('Resizing image to 60px width...')
const resizedImageBuffer = await sharp(originalImageBuffer)
.resize({
width: 100,
height: 100,
fit: 'inside', // This ensures the image is resized to fit within a 100x100 box
})
.jpeg({ quality: 70 })
.toBuffer()
console.log(`Resized image size: ${resizedImageBuffer.length} bytes`)
// Break into chunks if it's too big
const chunks: Buffer[] = []
const chunkSize = 100000
for (let i = 0; i < originalImageBuffer.length; i += chunkSize) {
const chunk = Buffer.from(
originalImageBuffer.subarray(i, i + chunkSize)
)
chunks.push(chunk)
}
console.log(`Number of chunks: ${chunks.length}`)
for (let chunk of chunks) {
console.log(chunk.length)
}
// --- 4. PREPARE DATA ---
try {
// We have to do these creates sequentially, as we need the returned hash to be used in the thumbnail (and additional parts if needed).
let creates_main: GolemBaseCreate[] = [
{
data: chunks[0],
btl: 25,
stringAnnotations: [
new Annotation('type', 'image'),
new Annotation('app', 'golem-images-0.1'),
new Annotation(
'filename',
req.body.filename || req.file.originalname
),
new Annotation('mime-type', req.file.mimetype),
...stringAnnotations,
],
numericAnnotations: [
new Annotation('part', 1),
new Annotation('part-of', chunks.length),
...numericAnnotations,
],
},
]
console.log('Sending main:')
console.log(inspect(creates_main, { depth: 10 }))
const receipts_main = await client.createEntities(creates_main)
let hash = receipts_main[0].entityKey
console.log('Receipts for main:')
console.log(receipts_main)
entity_key = receipts_main[0].entityKey
// Now if there are more chunks for the larger files, build creates for them.
// Start at index [1] here, since we already saved index [0]
for (let i = 1; i < chunks.length; i++) {
const next_create: GolemBaseCreate[] = [
{
data: chunks[i],
btl: 25,
stringAnnotations: [
new Annotation(
'parent',
receipts_main[0].entityKey
),
new Annotation('type', 'image_chunk'),
new Annotation('app', 'golem-images-0.1'),
new Annotation(
'filename',
req.body.filename || req.file.originalname
),
new Annotation('mime-type', req.file.mimetype),
...stringAnnotations,
],
numericAnnotations: [
new Annotation('part', i + 1),
new Annotation('part-of', chunks.length),
...numericAnnotations,
],
},
]
const next_receipt = await client.createEntities(next_create)
console.log(`Next receipt: (part ${i + 1})`)
console.log(next_receipt)
}
console.log('Sending thumbs and chunks:')
let create_thumb: GolemBaseCreate[] = [
{
data: resizedImageBuffer,
btl: 25,
stringAnnotations: [
new Annotation('parent', receipts_main[0].entityKey),
new Annotation('type', 'thumbnail'),
new Annotation('app', 'golem-images-0.1'),
new Annotation('resize', '100x100'),
new Annotation(
'filename',
`thumb_${req.body.filename || req.file.originalname}`
),
new Annotation('mime-type', 'image/jpeg'), // Our thumbnail is jpg
...stringAnnotations,
],
numericAnnotations: [],
},
]
const receipts_thumb = await client.createEntities(create_thumb)
console.log('Receipts for thumb:')
console.log(receipts_thumb)
} catch (e) {
console.log('ERROR')
if (e instanceof Error) {
if ((e as any)?.cause?.details) {
throw (e as any).cause.details
}
} else {
throw e
}
}
// --- 5. SEND A SUCCESS RESPONSE ---
res.status(200).json({
message: 'File processed successfully!',
originalSize: originalImageBuffer.length,
resizedSize: resizedImageBuffer.length,
tags: tags,
entity_key: entity_key,
})
} catch (error) {
console.error('Error processing image:', error)
res.status(500).send(
`An error occurred while processing the image: ${error}`
)
}
})
app.get('/thumbnails', async (req, res) => {
// todo: Consider building an index, as pulling back all the thumbnail data via query is a lot of unnecessary overhead
const thumbs = await client.queryEntities(
'type="thumbnail" && app="golem-images-0.1"'
)
res.send(
thumbs.map((item) => {
return item.entityKey
})
)
})
app.get('/parent/:thumbid', async (req, res) => {
let id: Hex = prepend0x(req.params.thumbid)
// Get the metadata
const metadata = await client.getEntityMetaData(id as Hex)
if (metadata) {
for (let annot of metadata.stringAnnotations) {
if (annot.key == 'parent') {
// Not sure yet, let's just return the parent key for now and see how that works
res.send(annot.value)
return
}
}
// No parent key found
res.status(404)
res.send('not found')
return
} else {
res.status(404)
res.send('not found')
return
}
})
app.get('/query/:search', async (req, res) => {
const results = await client.queryEntities(
`type="thumbnail" && app="golem-images-0.1" && tag="${req.params.search}"`
)
res.send(
results.map((item) => {
return item.entityKey
})
)
})
// Start server
app.listen(port, () => {
console.log(`Server is running at http://localhost:${port}`)
})
Ready for the front end? Let's go!
Head to Step 4.