Blog

Party Buzz Kill: modifying data

So Steve (SQL), Marsha (C), Bob (Python), and I (R) are at this party. We have TOTALLY cleared the room, especially now that Steve and I are deep into a debate about saving native data objects to disk versus storing data in a database.

Monica is a real person! She does consulting in Health Data Science. I don’t know if she serves punch.

I see Monica enter from the kitchen, carrying a bowl full of punch. It’s an awkward task and the fruity, sticky liquid is sloshing on the floor. Monica does data science, so I’m hoping she’ll come to my assist. Sure enough, she places the punch bowl on the table and joins us. She’s about to say something when the front door swings open.

Guenter walks in; he just got off a plane from Germany, so he looks a bit jet-lagged. Since the room is filled with a bunch of people talking SQL, he assumes database debates are the theme of the party.

Guenter is a real person too.

“I think I have already written an article in this context,” Guenter begins.

Look – It’s Helen Wall. She’s real too!

Before he can say anything more, Helen speaks up. “Perhaps talking about programming is an attempt to get everyone to leave the house at the end of the party so you can go to bed?” Where Helen appeared from is a mystery.

Monica listens for a minute, then interrupts the pointless debate between Steve and I. “People who are math aficionados” she says, “are a lot more comfortable generating datasets on-the-fly. People like me enjoy relying on the safety and reliability of importing a structured dataset we checked earlier!”

Steve is happy to hear someone is on his side. Steve thinks I’m a knucklehead. There are many people who agree.

“Sure, but there are advantages to not messing around with unnecessary overhead,” I say. “Let’s play with an example.”

I get out a new napkin and sketch out some R code…

Continue reading Party Buzz Kill: modifying data

MagPi Questions

Please tell us about yourself and your technical background as well as what got you interested in Raspberry Pi.  

I have a degree in industrial education, but it all started long before I graduated. My dad was a design engineer and brought home special projects. Super duper calculators. Electronics projects.

I’ve only taken a few formal programming classes and am mostly self-taught. When Raspberry Pi first came out, I was working for LinkedIn and decided to create a class on how to use and program this single-board computer.

What were the project aims? Please describe its capabilities and Raspberry Pi’s role (and any limitations you’re trying to overcome or extra functionality you intend to add). 

My urban garden needs irrigation during the summer months. The simple solution is to use commercially available irrigation timers; they turn on, then turn off. Simple enough. They cost about thirty dollars.

If you forget to bring them in during the winter, they freeze and break. After a few years, the plastic valves wear out and they jam closed (bad for plants) or jam open (bad for water bills). They don’t adjust to rain or hot weather.

You can buy more robust timers. Some of them have Internet connectivity and look up the weather forecast. They cost about $100. If you leave them outside they freeze and break.

Or… You can use your knowledge of raspberry pi and the assortment of parts in the electronics box to build your own.

Why did you choose to use a Raspberry Pi for this project and which Raspberry Pi are you using? 

I chose a Raspberry Pi for three reasons:

  • I need internet connectivity if I am going to look up weather reports.
  • My preferred language is R, which I can run from a linux operating system, but not from micropython or C
  • I have several Raspberry Pi’s gathering dust in my office. I should use them.

I am using a Raspberry Pi Zero WH. The wireless makes it easy to connect to the internet and the headers provide a convenient way to connect relays and buttons.

Please tell us about any other Raspberry Pi projects you’ve designed or ones that inspired this one. 

I’ve taught the LinkedIn Learning Raspberry Pi Essentials course and designed several projects for instruction and to demonstrate various sensors, servos, and switches. I’ve also embedded a raspberry pi in a Big Mouth Billy Bass.

Were there any particular challenges/advantages in designing it? 

This project requires a mix of a lot of parts:

  • The Raspberry Pi controller
  • Designing the logic to convert the desired amount of water balanced against rainfall (measured in inches) into seconds to hold the valves open to deliver irrigation water (measured in gallons per hour)
  • Programming the logic in R
  • Controlling relays to control the voltage required by the irrigation valves
  • Plumbing the flow of water from house lines through irrigation valves and out to the drip irrigation system

Is the design based on an existing project or did you have to design and create some or all the parts yourself? 

Entirely my own making. No plans needed, but there is a bit of experimentation required. Prior knowledge of plumbing and electronics is helpful.

How/where did you source the parts and software for it? 

All the parts are available at your local hardware store-or at adafruit.

Please detail the experience of setting up the project using Raspberry Pi. We will need at least three photos to illustrate the setup/assembly process please. 

I’ve documented the existing tool at https://github.com/mnr/sprinklR/wiki . This includes lots of photos

There is a dashboard showing performance at https://niemannross.com/sprinklR

This photo shows the development board. It’s just a piece of scrap wood with components temporarily screwed in place. At the top is the power for Raspberry Pi and irrigation valves. In the middle is the Raspberry Pi Zero, below that is the two servos used to control the irrigation voltage, and below that is the irrigation valves.

A jumble of electronics on a wood board

There isn’t any water connected to the system at this point – I’m only trying to test the electronics and develop and test the code. In particular, I’m trying to see if the software & hardware are providing realistic results. I don’t want to give it access to my water bill until I’ve reality checked its performance.

Here is an overall picture of the three parts. Left: Irrigation valves and plumbing, Center: Raspberry Pi and relays, Right: power.

Notice I’ve moved the project to a larger and more formal board. This shape was more suited to the layout of the parts. I wanted the plumbing to be physically below power and the Raspberry Pi in case there was a leak.

This is a closeup of the raspberry pi. I’ve housed this in a mason craft food preservation container. Costs about $3 and is waterproof – hopefully something I don’t need.

The base of the container is attached to the board chassis and has a hole to allow feeding wires through a notch in the base and up to the RPi. Access is by pulling off the jar – it’s held on by a gasket. Because the RPi Zero uses so little energy, cooling is not a problem.

Did you have to go through several iterations in order to get it just right and what did you have to tweak, if so? 

I proceed slowly, testing prototypes of each step. I did the coding on the RPi before it was hooked up to anything. I ran it with the relays for days without any valves connected. I tested the plumbing in a sink before I attached it to the board. So yes – I interate constantly.

What software/code does it run on and has the setup changed as you have refined it over time?

I run this on linux – my programming language of choice is R. The code constantly changes – thank god for git and github. I do a lot of development on a Macintosh, then push to github. I ssh into the Raspberry Pi and pull from github to update the running code. This gives me a slight buffer from the online system, and github provides me with a backup should the memory chip in the RPi fail.

Is the software available on GitHub or another open-source site? Any links you can share here would be great! 

  • Github Repository: https://github.com/mnr/sprinklR
  • Wiki documentation: http://niemannross.com/link/irrigation
  • Support for R access to Raspberry Pi GPIO: https://github.com/mnr/rpigpior
  • R for the Raspberry Pi family of computers: https://r4pi.org/

How much did it cost to build and how long did it take? Are there any significant running costs? Were there any particular challenges or unexpected hurdles?

  • Raspberry Pi Zero WH: $10?
  • Relays: $7 x 2 = $14
  • Irrigation Valves: $15 apiece
  • Random PVC: $25?
  • Power box, transformer, assorted parts: $25

How has using Raspberry Pi been beneficial to the project?

I wouldn’t have attempted this unless I was familiar with Raspberry Pi

How else do you think the ideas in the project could be used (if at all)? 

What advice would you give to someone who wanted to embark on a similar project? 

Think in small steps. Know where you are heading, but build the easy stuff first, then build up in complexity.

What hardware will readers need if they want to make their own?

Do you plan to do more with this project/with Raspberry Pi?

Finally, please tell us five fun/interesting facts about you or the project. 

  1. My next project is automating a chicken coop. Each chicken will get an RFID tag
  2. After that, I hope to build a homebrew GPS system for sea kayak navigation
  3. I write Science Fiction. My latest novel is “Stupid Machine” – a murder mystery solved by a refrigerator
  4. I have only taken one formal computer programming class in my life
  5. I have a degree in Industrial Education

After Matthew. Before Port Townsend

Janell and I first saw the canoe on July 4th, 2023. We were in town for a few rushed days between paddling the Apostle Islands in Wisconsin and paddling the Sunshine Coast in Canada.

Robin, Matthew’s mom, told us about the canoe Matthew started in high school and left in the garage. It was time for this boat to transition. Either finish it, or burn it. Just get it out of the garage.

We visited the boat between other getting-ready-to-go tasks and could see the shape of an emerging canoe. We noticed it was attached to a substantial strong back. It was not something we could easily move.

We told Paul, Matthews’ dad, that we were interested, couldn’t take it right now, and would have to figure out some logistical issues. We walked back to the car, excited but puzzled.

Continue reading After Matthew. Before Port Townsend

Rain – Evapotranspiration = mm Water

“Eeee-VAP-oooo-TRANS-PURR-ation,” I savor the word as I release it into our conversation. I’m still at the party with Marsha and Bob. We’re trying to determine why anyone (such as me) would want to use R on their Raspberry Pi.

Photo by Tima Miroshnichenko

“Big word,” says Bob. “What’s it mean?”

“Water evaporation from the earth and transpiration from plants,” I respond. “It’s a sum of the water escaping from my irrigation system. Look it up on Wikipedia.”

Marsha interrupts grumpy Bob; “So – That means, um…desired amount of water – rainfall + evapotranspiration equals the amount of water your irrigation system needs to supply.”

“Precisely,” I agree. “Until I found out about evapotranspiration, I was unsure how to account for temperature. I knew hot days would require more water because of increased evaporation; but was stumped how to translate temperature into increased inches of necessary water.”

“Never heard of it,” says Bob.

“Me neither,” I agree. “Evapotranspiration is handy, but doesn’t show up in all weather forecasts. Open-Meteo makes it available.”

“Say you’ve got seven days worth of this miracle number,” says Bob. “What does the R code look like?”

Continue reading Rain – Evapotranspiration = mm Water

Boat Lines

If you’ve spent significant time around boats, you develop an appreciation for the lines of a boat. I’ve shown the canoe to a lot of people, and at some point, people who know boats will walk to the bow, squat, close one eye, and sight along the keel. They assess the line from bow to stern—is it straight?

Wood strip canoes start with a strongback, a sturdy spine and stations to form the shape. If the strong back isn’t straight, then the lines of the boat will be off. 

Matthew built the strongback and laid the first strips at the Benson high school wood shop in 2013. He took pictures of his work, and it’s easy to see why the canoe looks like it does.

Continue reading Boat Lines

Party Buzz Kill: Data Storage

I’m at this party where Bob and Marsha and I are discussing the best languages for programming a Raspberry Pi. Bob advocates for Python, Marsha is a devout student of C. I’m defending my use of R. After all, Raspberry Pi starts with R. We have chased all the other guests out of the room with our conversation.

“With R, I have all sorts of built-in data management,” I say. “Manipulating matrices is in R’s basic DNA.”

Steve wanders in from the other room and joins our conversation. “Matrices aren’t a proper data strategy. You should be using a database. You can run SQLite on a Raspberry Pi with hardly any effort.”

Bob and Marsha simultaneously turn to stare me down. They are curious about how I’m going to get around this supposition.

“Sure. SQL with R–in particular SQLite, would have been easy to implement,” I pontificate. “Just call up RSQLite, push a few buttons, and Bob’s Your Uncle.”

“And that’s not what you did?” Steve is incredulous.

“I store the R object on disk and pull it into memory when I need it.”

“What kind of knucklehead stores data as a file on disk?”

Continue reading Party Buzz Kill: Data Storage

R Waters My Garden

I’m at a party, and the topic of programming languages comes up. A quarter of the room politely leaves, another half rudely leaves, and the remaining three people banter about the proper language for a certain project. Bob, Marsha, and I have the room to ourselves.

Tonight we’re talking microcontrollers: Arduino or Raspberry Pi. We like to connect things and aren’t afraid to release the magic white smoke with an ill-advised application of excess voltage. Bob assumes programming is done with micropython. Marsha prefers C.

“I use R to water my garden,” I say. “Yep, I installed R on a Raspberry Pi and use it to turn the water valves on and off.”

Continue reading R Waters My Garden

Booaat Camp

Before we go any further, That’s not a typo; that’s the way I pronounce the word “Boat.” I’m from Minnesoota, where all double vowels have an elongated pronunciation. Think of saying “ooooh!” like you just ate the best nooodles. So this week we’ve been in “Boat camp” although it might sound like “Boooot Camp.”

Continue reading Booaat Camp

The Church of the Wood Boat

This morning, we gather at The Chapel of Carpentry and Canoes for Sunday services. We’ll follow the Presbyterian Order of Worship. If we were to celebrate a Catholic Mass, we’d have to drink wine, a bad idea when using table saws and power tools.

Matthew has given up sawdust for lent so he chose to wander the city of Port Townsend. He will return to New York later today and is trying to remain vapor and sawdust free for the sake of his fellow passengers.

Janell, Rich, and I get breakfast at home then head to the boat shop. We avoid driving by the bakery in a vain glorious show of self-control.

Continue reading The Church of the Wood Boat