Posted in raspberrypi, aws

header_lq.jpg

Did you ever have a lot of data in your code that sits better ina file? Why not trying something else?

If I can, I always avoid storing information locally, especially when I need-to change something remotely. Recently I made Conway's Game of Life and I want the patterns to be accessible by other devices. This will allow me to create new patterns, store them and reuse them as-they are added.

For any thought/question/problem, use the comment section and tell me your thoughts on this. 

What do I need to get started?

If you never had any contact with AWS (Amazon Web Services), you can create an account for free at aws.amazon.com. 

DynamoDb it's a NoSQL database where you can store and retrieve data along with other capabilities. The best part, you can use 25Gb where you can perform 25 reads and writes per second for free. If these values get deprecated, just take a look at what free tier has to offer.

In other to be able to save and get items from DynamoDb we need to have the right permissions to access the AWS from Raspberry Pi. Go to the AWS Console and create (or use an existing user ) and attach the following policy to it, DynamoDbFullAccess. If you create a new user, don't forget to save the access key id and secret access key, we need them later. That's it the only thing we need to do from the AWS Console.

Take me to the code

Before jumping directly into the code, I will shortly describe what we will do. First, I will integrate DynamoDB into one of my old project, Game of Life. However, you can apply the concepts to any other project. 

I divided my code into two parts. The first one is responsible for seeding the table with data. The second one, as you suspect, it is responsible for reading the saved information.

Below, don't forget to replace the values for AWS_ACCESS_KEY and AWS_SECRET_ACCESS s

Seeding the table

This part can happen from any device you want, your PC or any other board that comes in handy.

This part is very simple, and you can see it from the code. We just need to create a client for DynamoDB using boto3. After we managed to do this, we have to create the table and save the data in it.

Because of my project, I need to save matrixes. Because DynamoDb does not support matrixes (I don't know if any database supports them), I need a serializer/deserializer. I didn't want to pollute the code with my own version, so I used pickle instead. 

I created three patterns for Game Of Life and stored them GameOfLifePatterns table.

import boto3
import pickle

AWS_ACCESS_KEY = 'REPLACE_ME'
AWS_SECRET_ACCESS = 'REPLACE_ME'
TABLE_NAME = 'GameOfLifePatterns' def createTable(client): table = client.create_table( TableName=TABLE_NAME, KeySchema=[ { 'AttributeName': 'pattern-id', 'KeyType': 'HASH' } ], AttributeDefinitions=[ { 'AttributeName': 'pattern-id', 'AttributeType': 'N' } ], ProvisionedThroughput={ 'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5 } ) # Wait until the table exists. client.get_waiter('table_exists').wait(TableName=TABLE_NAME) print "Table with name %s created.\n" % TABLE_NAME def putPattern(client, id, pattern): response = client.put_item( TableName=TABLE_NAME, Item={ 'pattern-id': { 'N': id }, 'pattern': { 'S': pickle.dumps(pattern) } } ) print "Item with id %s and pattern %s saved.\n" % (id, pattern) if __name__ == '__main__': try: dynamodbClient = boto3.client( 'dynamodb', aws_access_key_id=AWS_ACCESS_KEY, aws_secret_access_key=AWS_SECRET_ACCESS, region_name='eu-west-1' ) # just delete/comment this line if you alredy have the tabel # and you need to just save new things createTable(dynamodbClient) pulsy = [ [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 1, 0, 0], [0, 0, 0, 0, 0, 1, 1, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 1, 1, 1, 0, 0, 1, 1], [0, 0, 0, 1, 0, 1, 0, 1], [0, 0, 0, 0, 0, 1, 1, 0] ] toad = [ [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 1, 1, 1, 0, 0], [0, 0, 1, 1, 1, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0] ] beacon = [ [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 1, 0, 0, 0, 0], [0, 0, 1, 1, 0, 0, 0, 0], [0, 0, 0, 0, 1, 1, 0, 0], [0, 0, 0, 0, 1, 1, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0] ] # and many others putPattern(dynamodbClient, "0", pulsy) putPattern(dynamodbClient, "1", toad) putPattern(dynamodbClient, "2", beacon) # close execution by pressing CTRL + C except KeyboardInterrupt: print("Intrerrupted by user") pass finally: print("Program stopped")

I saved the above code in a save-patterns.py file, run it to save the patterns:

python save-patterns.py

Let's use the stored data

The data can be used for almost any device, as long it supports a connection to the internet. 

For my project, I just replaced the hard coded pattern (stored in a variable) with a method that retrieves it from the DynamoDB table. I chose to randomly use one pattern from the DynamoDb.

import time
import boto3
import pickle
import random

from luma.led_matrix.device import max7219
from luma.core.interface.serial import spi, noop
from luma.core.render import canvas

AWS_ACCESS_KEY = 'REPLACE_ME'
AWS_SECRET_ACCESS = 'REPLACE_ME'
TABLE_NAME = 'GameOfLifePatterns' UNIVERSE_WIDTH = 8 UNIVERSE_HEIGHT = 8 def isAlive(cell): return cell == 1 def drawMatrix(device, universe): with canvas(device) as draw: for y in range(UNIVERSE_WIDTH): for x in range(UNIVERSE_HEIGHT): fill = 'white' if isAlive(universe[x][y]) else 'black' draw.point((x,y), fill=fill) def countNeighbors(universe, cellX, cellY): count = 0 neighborsRadius =[-1, 0, 1] for x in neighborsRadius: neighbordPositionX = cellX + x if neighbordPositionX >= UNIVERSE_WIDTH or neighbordPositionX < 0: continue for y in neighborsRadius: neighbordPositionY = cellY + y if neighbordPositionY >= UNIVERSE_HEIGHT or neighbordPositionY < 0: continue if cellX != neighbordPositionX or cellY != neighbordPositionY: count += universe[neighbordPositionX][neighbordPositionY] return count def gameOfLife(universe): # copy the universe so that we can modify it without worry newUniverse = [row[:] for row in universe] for cellX in range(UNIVERSE_WIDTH): for cellY in range(UNIVERSE_HEIGHT): neighbors = countNeighbors(universe, cellX, cellY) if isAlive(newUniverse[cellX][cellY]): # Any live cell with fewer than two live neighbors dies, as if by under population. # Any live cell with more than three live neighbors dies, as if by overpopulation. if neighbors < 2 or neighbors > 3: newUniverse[cellX][cellY] = 0 # Any live cell with two or three live neighbors lives on to the next generation. elif neighbors == 2 or neighbors == 3: newUniverse[cellX][cellY] = 1 # Any dead cell with exactly three live neighbors becomes a live cell, as if by reproduction. elif neighbors == 3: newUniverse[cellX][cellY] = 1 return newUniverse def getDevice(n, block_orientation, rotate): # create matrix device serial = spi(port=0, device=0, gpio=noop()) device = max7219(serial, cascaded=n or 1, block_orientation=block_orientation, rotate=rotate or 0) print("Created device") return device def getPattern(client, id): response = client.get_item( TableName=TABLE_NAME, Key={ 'pattern-id': { 'N': str(id) } } ) pattern = response['Item']['pattern'][u'S'] return pickle.loads(pattern) def getNumberOfItems(client): response = client.scan( TableName=TABLE_NAME ) itemCount = response['Count'] return itemCount if __name__ == '__main__': try: dynamodbClient = boto3.client( 'dynamodb', aws_access_key_id=AWS_ACCESS_KEY, aws_secret_access_key=AWS_SECRET_ACCESS, region_name='eu-west-1' ) # replace 1 with the number of devices device = getDevice(1, 0, 0) # get a random pattern from DynamoDb numberOfPatterns = getNumberOfItems(dynamodbClient) universe = getPattern(dynamodbClient, random.randint(0, numberOfPatterns)) while 1: drawMatrix(device, universe) time.sleep(1) universe = gameOfLife(universe) # close execution by pressing CTRL + C except KeyboardInterrupt: print("Intrerrupted by user") pass finally: print("Program stopped")

I saved the above code in a game-of-life.py file, run it using the command below to play a random pattern from DynamoDb. 

python game-of-life.py

Conclusion

Storing data in a different storage than the local one is very useful. Especially when you have multiple devices access the same source of data. 

This can get very useful when you have devices that stores data from the sensor How-to be used later by other devices or even by an interface.

You managed to go through my poor writing skills, brave one. 

Comments

Be the first to post a comment

Post a comment