This post comes after some initial fun to work on interval scheduling, and it’s a good example of how you can start working on some initial problem, and creativity and curiousity takes you in an entirely different, fun direction! My idea was that, given that I could generate an interval randomly within some start and end time, what if I did that but updated the times to be within the range of the last generated set? It would mean generating a set of nested intervals. This design, I decided, was a bit like fireworks! But not just any kind of firework… it was a nested firework, and they come out beautiful:
TLDR I wound up going in a different and fun direction! If you want to see this post in action:
docker run -it vanessa/algorithms:fireworks
or forget an entire show, just generate one random “boum”
docker run -it vanessa/algorithms:fireworks --boum
That will automatically select between complex and simple designs. To force a simple or a complex design:
docker run -it vanessa/algorithms:fireworks --boum --simple docker run -it vanessa/algorithms:fireworks --boum --complex
You can also control the firework size:
docker run -it vanessa/algorithms:fireworks --boum --simple --size 5
Or the specific kind of
--complex design (try a random integer!):
docker run -it vanessa/algorithms:fireworks --boum --complex --design 5
or generate an entire fireworks show with one kind of complex design:
docker run -it vanessa/algorithms:fireworks --complex --design 5
Or watch (one of the infinitely possible) final shows!
First I want to review my algorithmic journey, because it was awesome and fun! If you are interested in adding fireworks to your clusters or code, skip to the Using Fireworks section, or jump to the end to see what I learned.
Recursive Internal Events
To be clear, this original idea did not turn out to be the final implementation, and I’ll comment on that later. It’s important that I describe my entire development process, and that starts with this first method. I summarized the problem as follows:
Let’s say that you start with a start value (a time in seconds, likely we would start at time 0.0) and end some N seconds later. This forms a range from 0 through N, and we would want to create some kind of fireworks display so that the fireworks come on gradually starting at time zero, climax in the middle, and then fade out. We would want nested events, and it might look something like this (s=start,e=end, and the number indicates the firework number!):
[start] [s1] [s2] [s3] [sN] .. [eN] [e3] [e2] [e1] [end]
I decided that I wanted to generate an algorithm to do the following:
- start with a number of fireworks, a start and end time
- randomly generate a firework, meaning design and size
- calculate trigger times for a firework depending on increasing size.
- create an event loop to trigger the fireworks.
Thanks to the magic of version control, you can see the first effort here.
Async.io for Asynchronous Events
I thought that I could build up intensity by adding more fireworks to be fired toward the “finale” of the show. Specifically, I had a Fireworks class that would let me generate multiple fireworks, each having a start time, end time, a design (color, character, size) and a calculated duration. The main function to run the show would then use async.io to create an event loop
import asyncio import time def fireworks_show(fireworks): '''create the loop of tasks to start the fireworks show! Parameters ========== fireworks: the schedule of fireworks! ''' print('Starting the show!') loop = asyncio.get_event_loop() start=time.time() tasks = [asyncio.ensure_future(booms(firework, start)) for firework in fireworks ] loop.run_until_complete(asyncio.wait(tasks)) loop.close()
The above took me a long time to do, because the concept of an event loop is completely new to me. I would then schedule each firework in the function called “booms” that you can see is handed to the loop as a task. The function would start a counter, and start printing the firework only after we had reached the start time or later:
@asyncio.coroutine def booms(firework, start_time): # Time passed since starting the show progress = time.time() - start_time # Sleep and hand off to other fireworks if not time to start while progress < firework.start: yield from asyncio.sleep(firework.frequency) progress = time.time() - start_time # When we are over the start time, calculate ending ending = time.time() + firework.duration # Keep firing until we reach the ending time while time.time() < ending: firework.boum() yield from asyncio.sleep(firework.frequency)
The cool thing about async.io is that with the expression
yield from asyncio.sleep
I am able to hand control to other processes in the loop, so one particular firework
doesn’t dominate control and not let others run. Although the above wasn’t
perfect (I might have used the loop’s time instead of
time.time() it achieved the
asynchronous quality that I was looking for. My fireworks were nothing to call
home about (just a line of a randomly selected character and color) but
the result looked like this:
If you had patience to watch the above for more than 30 seconds, other than being bored, you would realize that it does a poor job of actually modeling fireworks. Other than the visual representation being off, the timing is wrong too. No fireworks show that I’ve ever seen has a climax and then gradually comes back down. I think there are other real world things that follow this design, but fireworks aren’t one of them. And I was set on making fireworks! The more correct fireworks model looks like this:
[start] [s1] [s2] [s3] [s4] [s5][s6][sN] [end]
Notice the build up, represented by an increasing frequency? I needed to do that.
Fireworks in Ascii
But first, I wanted to deal with those epically ugly fireworks. I came up with a very simple algorithm to generate a design! The changes that I made are here. Overall, the algorithm was again pretty simple. Given some randomly selected characters and colors, along with a size and offset, I generated a design that is essentially two half circles. The function had a few different versions, but the more final one looked like this:
design = '' # Slightly different ranges determine top and bottom of shape top_range = range(2, size, 2) bot_range = range(size, 2, -2) ranges = itertools.chain(top_range, bot_range) for i in ranges: if i == size: i = size - 1 padding = " "*(size-i + offset) design += '\n' + padding + i*("%s%s" %(char1,char2)) return design
In the above, I’m going from 2 up to the randomly chosen size, and then back again, and adding a design character at some offset to form a circle with a bit of edge chopped off. This did slightly better at modeling something that is orby, but it still looked more like scrolling bubbles than any kind of firework. And when I added background colors to the random generation and a more consistent way to vary the speed, I got the following:
This looked really cool, but it wasn’t my goal to create scrolling bubbles and call them fireworks. It was in thinking about an algorithm to generate firework designs when things got fun :)
Firework Generation Algorithm
We needed a fireworks generating algorithm! I had two options here. I could generate ascii designs a priori and then modify the color (like I did with my salad fork generator, and this is easier, but less challenging) or come up with an algorithm to generate them for me. I spent the entirety of an evening with tabakg obsessively playing with this beautiful (and terribly documented) function:
import numpy as np def ascii_flower(char='O', n1=15, n2=15, inner=5, outer=15): S = '' for i in range(outer*2): for j in range(outer*4): x,y = i-outer,(j-outer*2)/2 angle = np.arctan2(x,y) Z = min((angle*n1/np.pi)%2., (-angle*n2/np.pi)%2.) r = np.sqrt(x**2 + y**2) if r-inner < Z*(outer - inner): S += char else: S += ' ' S += '\n' return S
When we made
n2, we got some beauuuuutiful designs. Like seriously,
this possibly beats my entire graduate education.
Humanizing the Algorithm
It would have been easy to use the above function, and stop there. But if someone
stumbled on it, they wouldn’t really get a sense of what is going on unless they are
very good with math, or test it out for themself. Unfortunately
I’m someone that has a learning requirement of huge verbosity (can you tell?), and
I would fall into the category of seeing this equation, and not
def choose_shape(self, char='O', n1=15, n2=15, inner=5, outer=15): '''generate a firework design based on inner and outer lengths, and variables n1 and n2 to control the angles. Parameters ========== outer: a multiplier for the outer radius. inner: a set inner radius "cutoff" to determine printing the char ''' design = '' # This is the (inside portion) of the outer circle (diameter) # [-----][-----] for i in range(outer*2): # This is the (outer portion) of the same circle (diameter) # [-----][-----][-----][-----] for j in range(outer*4): # We are scanning over values of x and y to define outer circle x,y = (j-outer*2)/2,i-outer # For each pair of x and y values we find the angle and radius. # arctangent returns radians, and reflects quadrant. # A circle goes from 0 to 2pi radians angle = math.atan2(y,x) radius = math.sqrt(x**2 + y**2) # We want a function of the angle to zig zag up and down. # We multiple the angle above by some factor (n1 or n2) # divide by pi and 2 so the result goes from 0 to 1 zigzag = min((angle*n1/math.pi)%2., (-angle*n2/math.pi)%2.) # Then from the angle we figure out the cutoff radius - # some value between inner and outer based on the zigzag # that we want. cutoff = zigzag*(outer - inner) + inner # (outer-inner) is the distance between outer and inner that # ranges from 0 (when zigzag is 0) to outer (when zigzag is 1). # This means that when zigzag is 0 it's scaled to none of the # distance, and when zigzag is 1 it's scaled to the entire # difference. # Compare the actual radius to the cutoff radius. # If actual radius is < cutoff radius --> fill in # otherwise put a blank space. if radius < cutoff: design += char else: design += ' ' # - At the end of a row, add a newline design += '\n' return design
You can read through the comments in the function and decide if I did a good job explaining it. Is it overkill? Maybe. I don’t think so. It may be the case that the computer is indifferent to the two versions, but for the human there are small changes so that it flows logically. For example:
People will compare the code to definitions they find I found that Arctan2 is defined for y and x. It’s an angle so ordering doesn’t technically matter, but I changed it to be consistent with the definition.
Lead up to complexity
As a reader, it’s natural to have basic variables defined before any more complex steps. I moved the definition of
radius to before
zigzag (the original variable
Z) so my brain would understand the simple stuff before getting tangled in figuring out
Break apart statements
It wasn’t totally intuitive what
if r-inner < Z*(outer - inner) was doing. By rearranging the variables and defining one ahead of time (
cutoff=zigzag*(outer - inner) + inner I was able to use the code to tell the user that the value was for a cutoff, and that the if condition was comparing it to the radius (
if radius < cutoff:)
Comments, ftw! I did not allow for any line to go unexplained. I challenged myself to justify every little thing, and it was a great learning experience. Yes, it’s a bit much. I’m ok with this.
This was very challenging for me, putting the entire thing into the right words. It taught me a valuable lesson about general programming:
The program runtime may be dependent only on the machine’s ability to “read” it, but it’s future development relies also on human digestability.
This isn’t just important for reproducible research, it’s important for general software engineering. Some future next dude should be able to look at your code and not want to poke hotdogs into his eyes.
At this point I was going nuts, I was so excited. I was also unhappy with the “flowing bubbles” effect that was being used to model the fireworks. I had great excitement to figure out the right character to use to clear the terminal. Figuring this out meant that I could clear the terminal between single fireworks, and have more of an “increasing in size and exploding” effect. This was another change to the algorithm. I had originally generated the design to be “hard coded” and stored with the Fireworks object, but realized that I needed to generate it on the fly, given some pre-determined variables like color and overall shape, and then vary the size. I also decided to use my “bubble” design as an center overlay on the more firework-y design:
Nothing anything weird? The center is kind of skewed. Unfortunately I spent a few hours trying to get this right, and ultimately decided it was simpler to keep the two shapes separately. My strategy was, instead of generating a long design string with newline characters, to put each full bytes sequence (the color and then off sequence) into a list of lists (a matrix) and then combine the two based on the center. The issue wound up being that, after the merge, the length of the byte sequences were slightly different (depending on the random generation) and didn’t line up perfectly as actual characters.
I also realized that the result was beautiful without the more complicated overlay, and by removing the asynchronous loop and just using a standard for loop, the fireworks looked much more like fireworks. And then I deleted a LOT of code! Here was another important lesson:
Simplicity is sometimes the best strategy.
In this case, I easily gave up many hours of work, and the result is a lot less code and a result I’m much happier with. More on this later. Let’s make some Fireworks!
We can start with a container and shell in to interact with the scripts (you can also use a Singularity container).
docker run -it --entrypoint ipython vanessa/algorithms:fireworks
singularity exec --pwd /code docker://vanessa/algorithms:fireworks ipython
or we can just clone the repository and use our own python. I tried to reduce dependency modules to not include any outside of Python 3 so this might work for you without a container, at least for now :)
git clone https://github.com/vsoch/algorithms cd algorithms/fireworks ipython
Single Firework Generation
Here is an example to create and explode a basic firework. Import the Firework class, instantiate it, and give it a start and ending time.
from main import Firework start = 0 end = 200 firework = Firework(start=start, end=end)
firework.speak() Baby I'm a Firework (00000:00200)!
Then what are you waiting for? Explode it!
You can customize your firework as you like:
firework = Firework(end=200, simple=False, design=13)
randomize to come up with a new design for the current firework.
Multiple Firework Generation
You could obviously put them in a loop:
# Or more fun... for i in range(0,10): firework.randomize() firework.boum()
And even set a specific design:
for i in range(0,10): firework.randomize() firework.thresh = 5 # star firework.boum()
But you can also have finer tuned control by using the firework’s internal generator,
the functions that are used behind
boum(). Here we will ready the explosion first,
and then iterate through the stages. The boum function adds a command to
clear the terminal, but if you don’t you get a different kind of experience:
import time show = firework.ready() for design in show: print(design) time.sleep(0.1) ''' kkkkk kkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkkkkkk kkkkkkkkkkkkkkk kkkkkkkkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkk kkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkk ... k kkkkk kkkkkkkk kkkkkkkkk kkkkkkkkkkk kkkkkk kkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkk kkkkkkkkkkk kkkkkk kkkkkkkkk kkkkkkkk kkkkk k kk kkkkk kkkkkkk kkkkkkkkk kkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkk kkkkkkkkkkk kkkkkkkkk kkkkkkk kkkkk kk ... k kkkk kkkkkkk kkkkkkkkk kkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkkkkkk kkkkkkkkkkkkk kkkkkkkkkkkkkk kkkkkkkkkkkk kkkkkkkkk kkkkkkkkkkkk kkkkkkkkkk kkkkkkkkk kkkkkkk kkkk k
Given the way that I’ve implemented “ready” the first design will actually be
the finished firework. This both adds to the explosion effect, and presents
the final firework first if the user wants it. You can change the implementation
ready() function to change this.
If you want to add the same clearing of the terminal, do it like this:
show = firework.ready() for design in show: print(design) time.sleep(0.1) print('\033c') # (-- how to clear the terminal
The ready function just generates an iterator over this function, you could change that too:
def ready(self): '''prepare the show! This is an interator to reveal slowly increasing in size fireworks. boum! ''' number = self.count_designs() delta = self.size / number inner = self.inner + delta for size in range(number): inner = self.inner - delta yield self.generate_design(size=size, inner=inner)
So arguably you can scrap the above and just call this function to generate a design!
design = firework.generate_design(size=20, offset=20, simple=False) print(design)
The sky is really the limit in terms of how you want to use this tiny script.
What I learned
While this originally was a project where I wanted to focus on learning about asynchronous event loops in Python, it turned out to not use the library at all. I’m okay with this, because I still learned about asyncio, and likely will use it for future projects. There were some subtle challenges and learnings that I would like to again highlight.
It’s sometimes easy to get caught up in wanting to use a new technology that you are just learning about, or that seems to be trending. It’s a more rare skill, whether you are a programmer or a designer, to remove elements to make it better. This fireworks algorithm isn’t perfect, but I found myself doing this at least twice, because I scratched asyncio, and more complicated overlapping designs. I like the result better for it. This general way of thinking should be applied to more things. Before jumping into the newest trend, you might step back and ask yourself, “Why?”
I’m clearly not one of those developers that goes for the “one line solution.”
While we obviously need to consider runtime optimization to some extent, I am a huge proponent of documentation and commenting being a standard in writing programs. If you do a thing twice, write a function and explain the inputs and what it does. This makes the code more likely to be used, run, and appreciated at some later date!
One of the reasons it took me a long time to do simple things like combining matrices and calculations is because I didn’t want to introduce any large dependencies beyond what is offered in standard Python 3.x. This means instead of using a library like numpy, I opted for the math module or operations on lists instead of traditional numpy arrays. Of course, things like this are always troubling:
Python why do you do this to me?
And while I can’t have confidence that the code will run on your computer natively, I can have (maybe?) a little more confidence that the container versions will persist slightly longer. Longer than that? Well it probably doesn’t matter. As I’ve said before, I’m not convinced that guaranteed, forever reproducibility is actually possible.
Exposure of Variables
The decision about what variables to expose to the user is not one to be taken lightly! I had
originally exposed the
start_time as a function variable, and this made sense given
that fireworks were generated in the “internal recursive” way I first imagined. However,
once I changed the algorithm and realized that, for the purposes of this little ditty, it would
be the case 99% of the time to have a start time at time zero, I decided to remove the variable from
command line exposure. It’s one less thing for the user to be confused about. If developers want
to have at it, it’s still available when calling the function.
You (and by you I mean I) don’t stop learning just because school is finished. I never learned much during those times anyway. In college it was all about World of Warcraft and learning about myself, and I loved graduate school but didn’t take away much from attending lectures, other than furiously writing things down with some hope they would be useful later. On the other hand, I know that I learn really well by trying to do something, and stumbling around until I figure it out. This way, I can almost be guaranteed that if I challenge myself with interesting problems, life is going to be very rich! Or at least if I don’t, I tend to get bored. I get very ornery and restless when I’m bored, I’d probably bite someone and then run up a mountain. Anyway, thinking about creative problems like this is one of my favorite things to do, after building things, and I imagine many others feel the same :)
If you are administrator of a cluster resource, you could use a Singularity container
to play some customized (or random) fireworks when your users log in to your resource(s) on the
Fourth of July. If you like to play with algorithms, you might want to take a stab at
overlaying fireworks for even cooler designs (and if you poke through my various commits, there is a good
start of this with the bug I mentioned). As the code is now, I set you up to do this!
You can add a
matrix=True to either of
return the same context in a list of lists (a matrix). Taking into account minimization of dependencies,
you would want to accomplish this without numpy.
Overall, I continue to be astounded by what relatively simple ideas plus some programming can come up with. This is what makes the practice such a rewarding thing to do. Rawr!