Matomo Google Analytics Importer to import our previous analytics
Some custom plugins we developed for custom post types like teaching and events are not yet open source. We are moving from Azure DevOps to GitHub for hosting repositories and these plugins will make the move soon.
Infrastructure with Terraform
The infrastructure repo contains a Terraform configuration to setup a blue-green deployment of a staging and production servers.
The setup is simple: a single Standard B1s server and a Premium SSD P4 32 GB for each environment. This costs about $30/month for two servers with one per environment. We only use about $360 of our $3,500 credits so the hosting is free.
Other infrastructure
We also use Azure storage for storing WordPress uploads and the Azure CDN for delivering uploaded assets. Uploads and CDN URLs are handled by the Azure Storage Plugin. Costs are minimal and there are plenty of left over credits to spend.
This storage infrastructure is not included in our web Terraform infrastructure because it was created long ago.
What’s next
As I need to recreate servers to upgrade them, I plan to extend the Terraform repo to include a wrapper stack to switch Cloudflare DNS to new servers. I do DNS changes manually for now.
Self-hosting is great
I hope this example of hosting a WordPress website will help you in creating websites of your own.
Nonprofits with nerds on staff (or volunteering) can make use of the $3,500 in Azure credit to make their website hosting free.
And if you are comfortable on the command line, you can use the Trellis CLI to create a new WordPress installation and start a $5 DigitalOcean droplet which is more than powerful enough to host most websites. You can locally develop your changes and deploy them to your production site with little effort.
Getting started looks something like this:
brew install roots/tap/trellis-cli vagrant virtualbox
trellis new example.com
trellis up
This very site uses Trellis. I host several more websites using similar tools which makes self-hosting WordPress much easier.
WordPress block editing has made WordPress into a very flexible tool for content. I recommend you try WordPress out if you haven’t used it in a while — it’s good.
In 2019, I built a new live streaming PC for Awakening Church. I am excited to share the details of our live studio PC and how you can build your own to have high-quality live production switching and streaming without the typical high cost.
A few months ago we shifted to recording our weekly broadcast in advance using DSLRs, but we’ll soon be returning to a live workflow and we will be using this equipment at the core.
Background
Back in 2016, Awakening Church used a single Blackmagic Pocket Cinema Camera with a Teradek VidiU to stream to our own self-hosted live video server. Times have changed. We now have a multi-camera production with live switching to make it easier to produce live streams.
Before our latest 2019 build, we used a custom computer built in 2017. This computer worked well but each camera required a lot of cables: power, SDI, and VISCA in/out. This made cabling heavy and cumbersome to setup. Additionally, the PC was in a standard case, which means we had to setup all cabling from scratch every week.
This computer was stolen in late 2019. We thankfully had insurance which covered the cost of building a replacement.
Awakening has been and still is a mobile church which needs to setup and teardown every week. When building the replacement, we kept setup and teardown issues in mind.
Requirements
Entire system easy to assemble and store
Operation by a single volunteer — this means remotely controlled cameras
We adopted NDI|HX cameras which can handle power, network control, and H.264 video all in a single Ethernet cable.
We no longer need a capture card for every camera, but we still need one for capturing ProPresenter slides.
We use a rackmount design to allow the computer to remain in a portable rack alongside a PoE switch and PDU. This lets us easily setup and teardown the system every week.
Here’s a picture of the portable PC rack:
The EdgeSwitch is for the camera network. The switch provides power to the cameras and a PTZ controller which makes it easy to move the cameras. You could also just use the PTZ controls within Livestream Studio to save more on costs.
Here’s the part list, including the switch, PDU, and Gator rack:
We already had a donated 150W EdgeSwitch, so the cost isn’t included in the above list.
In March 2020 we setup this equipment semi-permanently in our church’s studio. Here’s a photo of the early setup. We now have large 4K monitors instead of a single small one, but this small monitor worked well for easy storage in a Pelican 1650 case.
Hasty setup in the early quarantimes
Performance
This system performs well for a fraction of the cost of typical commercial live streaming systems. The Intel i7-9700 includes an integrated GPU with Intel Quick Sync video which hardware accelerates H.264 encoding and decoding in Livestream Studio.
However, this setup did not perform well when adding a third camera.
Late night preparing for Sunday at the ?@AwakeningSV studio?. Pushed our Studio PC to the limit and ran into thermal throttling. This stopped our live stream on Wednesday. Made lots of changes to keep the PC cool and make live services run smoothly. Production is hard work! pic.twitter.com/jge32PyEQr
After adding a third NDI|HX camera, we’re starting to hit the limits of performance of this system when also used with a computer HDMI input which requires conversion.
My friend Douglas suspected thermal issues with the CPU, so we looked into that as well, but the real issue appeared to be the need for a dedicated GPU to keep the CPU workload down.
Things are cooler now but there’s high load on the iGPU. Ordered a Quadro P1000 since having all 6 cores and the iGPU at full blast is a bit much. pic.twitter.com/dc2rFPv43L
We added a Quadro P1000 graphics card from B&H after running into performance issues. Livestream Studio appears to work much better with this card in place. Our previous CPU usage was very high — often over 85% during recording and streaming. We’re now back down to about 50% according to the operator who ran cameras today.
This added $317.99 to the cost of the above build.
I’d bet you could save some money and get a Quadro P400 to do the same job. It’s the same card used in Livestream’s own high-end HD51 4K edition which costs $8,999. (See Livestream product datasheet.) The P400 card is only $118.74 from B&H.
The integrated GPU is disabled in firmware by default because we added a dedicated GPU. I was curious if running both the iGPU and Quadro would be beneficial, but I didn’t bother turning it back on after the tests went well.
If your build is not constrained to low-profile cards due to a 2U rackmount design, you have a lot more options for hardware encoding and decoding with NVIDIA than the Quadro series, but this card does quite well and includes a regular-sized bracket for use in traditional PC cases. Since the P400 contains the same hardware H.264 decoding and encoding engines and stream limits as the P1000, consider the P400 before needing to add on more cost with a more expensive card.
Summary
The rackmount setup in a portable case is ideal for setup and teardown. Using NDI|HX takes more compute resources but really reduces the amount of cabling. And the machine is very easy to build with commonly available parts.
We are preparing to use this equipment again for live streaming in an outdoor setting under a tent. In the future, I look forward to sharing what we learn from adapting this equipment to our new environment.
Today I took a trip to the Pulgas Water Temple. It’s only open on weekdays, so I stopped by for a few minutes late in the day. I took these pictures on my Canon EOS 6D with my go-to 17-40 f/4 L lens.
This stone structure near Interstate 280 commemorates the completion of the Hetch Hetchy Water Project.
Water from the Sierra Nevada mountains can be heard rushing past a waterfall in the center of the temple. A reflection pool in front of the temple makes this a beautiful place to visit.
Inscribed on the frieze and above the waterfall is a wonderful reminder of God’s provision:
I give waters in the wilderness, and rivers in the desert, to give drink to my people. Isaiah XLIII:XX.
I originally wrote this post in late 2017. A lot has changed since then! Our original PC was stolen from a storage facility and we rebuilt it. I’ll be sharing our 2019 rebuild in an upcoming post.
In 2017, I helped build a Windows PC to run Livestream Studio for my church.
Livestream Studio is a proprietary live production video switcher which can accept multiple video inputs and screen capture video from a Mac over the network. An update earlier this year allows for hardware accelerated H.264 encoding which makes this system usable on cheaper processors.
Why Studio?
Studio’s new hardware acceleration means we can do multiple HD feeds on relatively cheap processors. Studio works well with a customized keyboard which is very nice. There are many competitors in this space, I believe you get what you pay for here.
Requirements
This is an unusual system with many requirements unique to our application:
The system must run Windows because Studio requires it.
The system must be powerful enough to run the CPU intensive Studio program with multiple video feeds.
The system requires two PCIe x4 slots with at least one more for expansion to run HD-SDI video capture cards.
The system requires a secondary NIC to receive video and audio from a separate laptop running ProPresenter.
The system should be affordable with a budget of $1,800 excluding the cost of Studio.
The system must be portable and fit into one or more Pelican 1650 cases. Our church needs to tear down all equipment every week.
The system must be quiet: it’s going to be used next to the seating area of the auditorium.
We’ll accomplish this build with the help of PCPartPicker which handles all of the compatibility issues between parts. I’m not going to consider parts which are not on this resource — the power of the tool for checking for compatibility issues is too valuable, especially with limited time.
Off the shelf: Livestream HD switchers
Livestream produces HD switcher PCs as a turnkey system. However, the PCs they produce are way out of our budget. They use Intel Core i7 processors of various generations.
I was about to purchase this system along with some used Blackmagic UltraStudio SDI external capture cards; however, I changed my mind due to a few troubling factors:
Livestream’s All-in-One guide really pushes this system to its limit and it’s accepting 3 HD streams as configured with typical CPU load of 60%. This is okay but it means the system will run hot.
The nice UltraStudio SDI devices are discontinued. Magewell produces a similar device for SDI capture, but reviews state the SDI port is difficult to use being recessed into the case. This is not good for repeated abuse when setting up and tearing down 50 times a year.
We’re looking for something like a Micro ATX form factor. The Cooler Master Silencio 352 was selected for its sound dampening material, low cost, clean interior, and dimensions that tuck nicely into a Pelican 1650. Full ATX would be too large to be portable, so this removes a lot of motherboard options.
Ah, to ECC or not to ECC. This is supposed to be a reliable system — a crash would cause an outage of our live video. So, I explored using ECC RAM.
ECC RAM means you must skip to Xeon processors. If we want to use Quick Sync, this leads us to a Xeon E3 line of processor. The fastest E3 according to cpubenchmark.net is the Intel Xeon E3-1275 v6 @ 3.80GHz, which is a modern Kaby Lake processor with a score of 11,445.
The trouble comes in finding a good motherboard. The Asus – X99-M WS Micro ATX LGA2011-3 Motherboard is one of the only Micro ATX boards available for this processor. While it includes attractive features, it does not support video connectors to take advantage of the embedded GPU for displaying video. We’d need a cheap graphics card which will consume a PCIe x16 slot. There are 4 slots, leaving us with two open PCIe x16 slots and a PCIe x1 slot. This is just enough.
In this setup, the additional cost doesn’t come from ECC RAM: it’s the graphics card and relatively more expensive motherboard. While the X99-M is one of the most reliable motherboards it’s extra cost will put a strain on meeting the budget and a plain i7 7700K gets a better CPU benchmark score.
Let’s forgo ECC in favor of a Core i7, which is used in Livestream’s official products.
Motherboard
We’d like to expand with a few extra PCIe x4 slots available. A few options work here. We picked the Gigabyte – GA-Z270MX-Gaming 5 Micro ATX LGA1151 Motherboard because of the open slots — it’s one of only a few boards that had room for expansion in this form factor. Despite the gaming styling, the brand is noted for reliability over competing Micro ATX offerings by other brands.
CPU
The Intel – Core i7-7700K 4.2GHz Quad-Core Processor hits the sweet spot of price and performance with a cpubenchmark.net score of 12,174. This is a bit better than a similar Xeon and significantly better than the Skull Canyon NUC.
An inexpensive IPS 1080p monitor was chosen to do the job. The monitor and its stand are thin enough to fit in the top foam of a Pelican 1650 and it’s very easy to tear down the monitor stand into flat pieces — a must for packing this system into a case.
Everything else
32 GB of memory, a 480GB SSD drive, and a reliable EVGA power supply round out the build.
Prices include shipping, taxes, rebates, and discounts
Total (before mail-in rebates)
$1,828.41
Mail-in rebates
-$10.00
Total
$1,818.41
Generated by PCPartPicker 2017-07-02 01:26 EDT-0400
Assembly
The parts arrived a few days after ordering.
A team of people from my church helped assemble the PC.
This was our first time assembling a computer, so we took our time. But it was all fairly simple.
Even though the internals will not be on display, clean cable management was important. The cables needed to stay in place despite lots of transportation jolts.
The final result was very clean and featured the Livestream USB license dongle mounted internally in the case.
Deployment
Before deploying the system, every component was tested.
The end product fit inside two Pelican 1650 cases. The recording HDD fit inside a separate hard case because the drive would be transferred frequently from our location to another location for video editing.
The computer case and cameras fit into one Pelican 1650. Everything else, including all wiring, controllers, keyboard, and the LCD monitor all fit into the other case.
The system was deployed and thankfully everything worked!
Five months later
We made a few improvements after the first use of this system.
Camera support in the middle of a theater
We initially planned to place the cameras on high ledges in the theater. Instead, we opted to place one camera on a lower ledge and the other on a stand. A traditional tripod would not work for the space, so we used a OnStage studio monitor stand. We used velcro on the camera and monitor base to keep it from moving or falling off and a sandbag on the base to prevent the stand from tipping over.
After a few weeks, one of the plastic feed on the bottom of the stand fell off. This caused the camera to be slightly off-level and took me a while to figure out what was causing it.
Loose cabling
A few weeks after the initial deployment, one day the computer failed to turn on. I opened the case and everything looked okay. After pulling gently on the modular power supply’s cables, I noticed the CPU power connector became loose. It has not become loose again after re-seating it, but I’m glad I was the one volunteering that day!
Cable management
I initially purchased a “cable dropover” to hide the camera’s SDI and power cables across the floor in a very high traffic area. However, the dropover was difficult to use and store. We instead switched to a Safcord cord protector which hooks to carpet and is very easy to setup and teardown. Safcord is awesome!
We also had long cable runs between cameras. I used some cheap generic Kootek neoprene cable management sleeves which were very easy to use and keep everything tidy in a long cable snake between cameras.
Color consistency
We noticed our PTZOptics cameras tend to acquire a color cast depending on the graphics we display on our projector. To work around this problem, I manually set Auto White Balance off and manually set color temperature on both cameras. This makes our color consistent even as the background and stage lighting changes.
Livestream software
Everything pretty much works. With two 1080p cameras via SDI, a virtual network camera capturing audio and video from ProPresenter, occasional media playback of videos presented during services, and various graphics layers to show sections of the slide display for lyrics, the computer handles all of this without any problem. CPU usage is normally around 30% and can get as high as 70% or so.
We’ve never had a software problem which impacted our broadcast, which is great.
I do notice some visual glitches during complex transitions, and some keyboard focus issues which can cause keyboard hot keys to fail to work until I click and re-focus the UI. These issues are odd for professional and expensive software, but they are minor and easy to work around.
Conclusion
Overall, this was a very cost effective build and served us for a long time. This computer was stolen in 2019 and we rebuilt a rackmount version which I’ll write about in the future.
Last Sunday was an unforgettable day riding through SF and across Golden Gate. We rode united to fight male suicide and prostate cancer and had a great time doing it.
Why I am asking for your support
Men are dying too young.
Prostate cancer kills more men than breast cancer in women.
I lost a male co-worker to suicide, which is the biggest killer of men aged 20-39.
This needs to change.
That’s why I have been partnering with these fine folks riding for a cause — the Distinguished Gentleman’s Ride — to raise funds that directly support the fight for men’s health worldwide.
Prostate cancer statistics
Prostate cancer is the most common cancer in men. It kills more men than breast cancer in women.
One in seven men will develop prostate cancer by the age of 75, and one in four men will be affected by the age of 85.
Close to 500,000 men will die from prostate cancer yearly.
Suicide prevention
Suicide is the biggest killer of men aged 20-39, with ¾ of all suicides being male.
510,000 men die from Suicide each year, one every minute.
These numbers need to drastically change.
Personal fundraising goal reached
I have had a few fine folks give generously before the ride: $522.40 contributed for men’s health. Thank you. You have personally contributed in funding worldwide medical research to find a cure for prostate cancer, as well as aiding to improve men’s mental health.
$1.5M to go worldwide
Fundraising is still open until October 31. If you haven’t yet donated, please consider registering your support.
All fundraising has only been made possible through the generosity of people like you who have answered the call to make the world a better place. DGR has a goal of $5M and $3.4M have been raised so far. Help us get to the end!
The ride was pretty incredible. Over 350 people turned out. Together we raised $55,511.
This short video gives you a taste of what it’s like:
After riding across the Golden Gate we stopped at Equator Coffee, where we all took some time to drink cold brew, recognize the top fundraisers, and even the best dressed gentleman and gentlelady along with best motorcycle. And, I took home an Iron & Resin t-shirt as a runner up for best beard. Tip of the hat to Riv at Paradox for keeping me looking sharp.
