Hosting your Static Site with Amazon S3

If you followed the previous post about getting started with Pelican, you should have a Pelican website up and running on your local machine. This is where a lot of web development tutorials stop and that has always frustrated me. There are a million and one places to learn how to code online but just because you learned how to write a for-loop doesn’t mean you can actually make anything.  This post is meant to help bridge the gap between “starting a web project” and starting a web project on the actual web.

The goal of this post is to get your collection of HTML files that you’ve built using  pelican content uploaded to Amazon S3 so that everybody can access your site on the web!

This is the Amazon S3 Console. We’ll get to this soon…

Why host on Amazon S3?

In the last post, I discussed why I chose to build a static site with Pelican and there was a similar set of considerations why I chose to host this project on S3. I will address the main two:

Why not Github Pages?

Github Pages is, of course, awesome. There is no easier way to host a static site on the web and best of all it’s free. So why didn’t I choose the path of least resistance? The answer is configurability and server access logs. GitHub Pages definitely favor simplicity over configurability, and as a result, don’t give you many options for tweaking your site. That’s great for most but not for exploring the technical side of SEO where logs are really important.

Why not Netlify?

Netlify is also very cool. Like GitHub Pages, Netlify allows you to deploy sites from GitHub. It also strikes a good balance between simplicity and configurability—leaning more to the configurable side than GitHub Pages. It also has a host of very cool features, many of which are available for free. If I were just doing this as an ordinary web project, I probably would have chosen Netlify, but because this is meant to introduce more of the bare metal concepts, AWS wins out.

On top of those questions, there are really good reasons to choose AWS in its own right:

  1. It’s huge. So many companies host their sites and apps that it’s worth getting familiar with their concepts and terminology.
  2. It’s huge. AWS has so many services that you can take advantage of. We’re only taking advantage of S3, Route 53, and CloudFront but starting with AWS makes it easy to scale your projects if you want to do something crazy.
  3. It’s huge. The Cloudfront CDN is among the best out there and it allows us to mess around with HTTP headers and send the server access logs to an S3 bucket so they are as easy to access as our site.

On the flip side, AWS (and Amazon) is huge. So that may be a consideration for choosing other hosting solutions or CDNs. There are lots out there to choose from. I’d say just google them, but if you’re not into big corporations, I’d say, go try DuckDuckGo.

Prepare your static site for the cloud

Luckily, I don’t need to reinvent the wheel much here. @Kent put together an excellent technically-focused walkthrough. The only difference between this tutorial and his is that we are going to use Route 53 instead of a Cloudflare.

Up to this point, you should have a /output directory that has a handful of HTML files after running  pelican content. You could put an article in your /content directory to generate your first article. For instruction on adding an article,  refer to the Pelican articles docs.

That said, you don’t need any articles yet to keep pushing forward to get your site live.

Setting up relative URLs in your pelicanconf.py file

You may have already modified your pelicanconf.py file in order to apply your Pelican theme. If you haven’t it’s time to make your first modification (and a very important one).

What is a relative URL? you might ask. That’s a good question and an important one when it comes to hosting your site on the web. Relative URLs are URLs that start with the URL path rather than the protocol (https://) and the hostname (www.yoursite.com). In other words, they look like this: /my-folder/some-page/.

Why is that important? We are going to move all the site’s pages your computer, (http://localhost/) to an S3 bucket with a URL like yoursite.com.s3-website.us-east-2.amazonaws.com/, and in the future to your chosen domain name (something like yoursite.com). If the site refers to internal pages by relative, instead of absolute URLs, you won’t have to worry about every internal link breaking every time you change your hostname. (This is also really important when it comes to domain migrations!)

 # Uncomment following line if you want document-relative URLs when developing
 # RELATIVE_URLS = True

By default, Pelican will prefix all your URLs with http://localhost:8000 when you’re building your site locally. In order to change this to relative URLs, there is an easy switch in your pelicanconf.py file. All you have to do is find these lines and uncomment the line that says  RELATIVE_URLS = True.

Setting up your Amazon S3 Bucket

S3 stands for “Simple Storage Service.” The concept is quite simple, S3 provides you with “buckets” to host your content on their servers. It’s great: there’s no need to configure or manage a server.  Buckets can be public or private but for our purposes, we’ll have a public-facing bucket. 

Uploading a site to an S3 bucket is pretty simple but first, let’s set up an S3 bucket. You will, of course, need to set up an AWS account. If you don’t have an account yet, you’re in luck. You can start up your account for free for one year!

Creating your first S3 Bucket

Once you have an AWS account, follow these steps to create a bucket.

  • Go to your S3 console: https://console.aws.amazon.com/s3
  • Click the “Create bucket” button
  • Name your bucket after the domain name you plan to use. My project is going to live at techdefs.com, so I named my bucket “techdefs.com”.  

If you haven’t decided on a domain name, now’s a good time to head over to Amazon Route 53 and find one that’s available.

  • Select a Region where you would like your content to be hosted… You could choose one near where your audience is likely to be, or you could choose based on price. I chose US East (Ohio) but to be honest, for this project, it doesn’t really matter.
  • Un-select “Block all public access”.This will allow your website content to be accessed publicly on the web.
  • Click “Create Bucket” at the bottom of your screen to finalize your bucket.

Configure your S3 Bucket to host your site

  • Open the bucket Properties pane
  • Choose “Static Website Hosting
  • Choose “Use this bucket to host a website”
  • Name of your index document in the Index Document box. For Pelican sites, the index (aka homepage) document is index.html.
  • Click Save to save the website configuration.
  • Copy your Endpoint URL and paste it somewhere for later use

Configure public access for your S3 Bucket

The last step will be to set the bucket’s security policy to allow public access (so everyone can view it on the web). Bucket policies determine who can access your buckets and what level of permissions they can have. For example, you might only want to grant view access to some people and write access to others. Read more about s3 bucket policies here.

For our purposes, we are going to allow “PublicReadForGetBucketObjects” for the objects (HTML files)  in the bucket that hosts the site. See step #3 below for more details.

Go to your new bucket and go to the Permissions tab

You should see “Block all public access“ is set to “Off”

  • Click Choose Bucket Policy
  • Paste in the following policy and replace yoursitename.com with you actual site’s name
 {
    "Version":"2012-10-17",
    "Statement":[{
   "Sid":"PublicReadForGetBucketObjects",
          "Effect":"Allow",
     "Principal": "*",
        "Action":["s3:GetObject"],
        "Resource":["arn:aws:s3:::yoursitename.com/*"
        ]
      }
    ]
  } 
  • Click Save

Congrats, you have your first S3 bucket! Now let’s fill it with your website!

Upload your site to your S3 Bucket

There are two ways to do this. One is easy but manual, and the other takes a bit more time to set up but automates the process for the future. Since we have spent most of the time on this post “setting things up,” first we are going to do it the easy way first so we can see the site live on the web!

  • Go to the Overview tab
  • Click Upload
  • Open up a Finder window (assuming your using Mac) and navigate into your /output folder
  • Drag and drop all the files in /output into the Upload box (not the /output folder itself)
  • Click Next to proceed 
  • Set “Manage public permissions” to “Grant public read access to this object(s)” and click Next 
  • Leave the Storage Class  set to Standard and on the “Set Properties” step and Click Next
  •  Click Upload on the Review step
  • Your files will be uploaded to S3 shortly

If everything has gone well up to this point, your site is ready to view! 

Paste your Endpoint URL from step #6 of the “Configure your S3 Bucket to host your site” section above.

🎉🎉🎉 Your site is (hopefully) live! 🎉🎉🎉

Troubleshooting

Maybe it didn’t work perfectly… That makes a good opportunity to learn!

If your homepage looks like CSS rules weren’t applied

View the HTML source of your page and check if the CSS links look right. (They might be pointed to localhost). Go back to the “Setting up relative URLs in your pelicanconf.py file” step and check that everything looks good.

If your homepage displays an XML error saying that you don’t have permissions

This is probably because you missed a step setting up public permissions. Recheck the “Configure public access for your S3 Bucket” step and the “Upload your site to your S3 Bucket“ step to ensure that your site has public permissions.

Moving on

You might be satisfied here… I mean you have a website up and hosted. As you add articles, you can upload them to S3 as you’ve done before. But… you don’t have a cool URL and the deployment process is still pretty manual. And we haven’t even done any fun SEO stuff.

In the next posts, we’ll set up a custom domain with Route 53 and set up an automated s3 upload process. Then, we’ll do some interesting SEO stuff.

Starting an SEO Project with Python, Pelican, and AWS

I’ve been thinking about the best way to put a “course” together to demonstrate the overlap between web development and SEO. There are a lot of directions this could go in but I wanted to strike the right balance between technical depth and feasibility for someone who hasn’t done much in the way of web development. 

This is the beginning of this so-called “course,” though it’s more of a guided SEO project. Though this is just the beginning, I hope to teach something about technical SEO and SEO analytics by playing around with website code and hosting infrastructure and different measurement tools. Hopefully, you’re interested in Python too 🙂

Launching an web development and SEO project

To start the project in the right direction, I had to determine what technologies to use that balance of technical complexity and ease.

I had some considerations:

  1. Should I choose WordPress? Sure, it’s popular, but there are already tons of WordPress tutorials out there but the last thing I want to do is tell people they should go out and learn PHP and tear at the internals of a 15+ year-old web framework. 
  2. Python continues to grow in popularity. And that’s awesome, but I feared that, if this project were dependent on the audience’s ability to  deploy a Flask or Django site, it would steer the focus away from SEO toward web development,
  3. What about Jekyll? A static site generator seemed like a good balance between simplicity and technical depth.  (They are also really affordable to maintain!) Jekyll seemed like a good option but I opted against it because it’s built on Ruby. And Ruby, like PHP, just isn’t as hot as Python these days. 

This meant the focus would be a static site generator based on Python. This made Pelican an easy choice. Pelican has been around long enough and garnered enough support to have a decent ecosystem and plenty of well-written “Hello World”  tutorials. 

How Static Site Generators Work

Static site generators are a good balance between the power of a full-blown CMS and the simplicity of a pure HTML site. With a static site generator like Pelican, instead of worrying about hosting an application and a database, you only have to manage “flat files” and host the fully-rendered HTML pages on a server or file store like S3. 

Most static site generators, work like this:

  1. Choose or develop a theme that determines the style and layout of your pages
  2. Edit your site’s content in markdown files locally on your computer
  3. Run a command to build all the HTML files that make up your static site
  4. Transfer these pages from your computer to the hosting service of your choice
  5. Your website is up and running!

This means this project can be more about demonstrating SEO levers than web development. 

Introducing Pelican: A Static Site Generator, Powered by Python

Pelican is conceptually pretty simple. At a high level, you can think of it like this: 

  1. Your content: The “flat files,” commonly markdown or reStructuredText files, that you write and update to generate content on your site
  2. Pelican configurations: The settings in pelicanconf.py and publishconf.py that Pelican refers to when building your site
  3. Pelican itself: The processes that reads your content and configurations and generate the HTML files that make up the complete static site
  4. Pelican themes: Either pre-built or custom-build, themes make up the page templates (based on Jinja), CSS, and Javascript files that create the look and feel of your site
  5. Pelican plugins: Add-ons that allow you to change how Pelican reads your content, outputs your site, or does just about anything else during the build process

That means if you want to modify your site, you basically have one of two avenues: modify your themes or add/build plugins. 

That’s really nice compared to WordPress, where you would have to think about a MySQL database schema, WordPress’ architecture, and… writing PHP to generate pages. This isn’t the answer for the next Yelp, but it will let you do some interesting things with SEO for .0000001% of the complexity!

Getting Your Web Project Started

With any web project, there is some groundwork to be done before the real construction begins. If you’ve done any kind of development project before, you’ll find most of this looks pretty familiar. 

If you haven’t done any development project, I recognize that getting started can be the most challenging part. Hopefully, these resources should be sufficient for you to get started with enough coffee, grit, and free time.

Setup your Development Environment

If you’re unfamiliar with a development environment is more of a state of preparation than an actual “thing.” A development environment means having all your necessary software, packages, settings, and tools loaded, working correctly, and understood generally understood.

If you want some guidance with this step, I suggest Peter Kazarinoffs’ guide to setting up a development environment for Pelican. In his first post he covers:

  • Installing Python 3
  • Setting up a virtual environment (to keep this project’s packages separate from other Python packages on your computer)
  • Installing Pelican and other required Python packages
  • Creating a GitHub.com account
  • Making a directory for the site and linking it to GitHub

That’s a great guide to follow if this is your first Python development project ever (and even if it’s not).

Getting started with Github the easy way

This project assumes that you’ve never worked with git or GitHub before. For that reason, the demonstrations will use Github Desktop and the Atom code editor because they take a lot of the complexity out of git. So when it comes to the “Making a directory for the site and linking it to GitHub” step above, I’d suggest following this video about creating a repository with Github Desktop

Getting your site running locally

At this point, you can find numerous tutorials to get your site up and running on your “local machine” (aka your computer). I think Matthew Devaney’s tutorial series is pretty easy to follow. To get your site running locally follow his tutorial about installing Pelican, choosing a theme, and running it locally. You can also continue ‘s tutorial for guidance on using a virtual environment and build automation

You’ve completed this project step once you have done the following:

  • Installed Python and set up a virtual environment
  • Installed Pelican and the required Python packages
  • Created a GitHub account
  • Created a directory for your Pelican project
  • Linked your project site to Github
  • Setup Pelican via pelican-quickstart
  • Successfully ran pelican content to generate your pages locally
  • Opened your bare-bones site in your browser after running pelican --listen

If you’ve made it this far, congrats! You’ve made it past one of the hardest parts of this whole project.

Up next: Play around with Pelican and see if you can generate some posts. Our next step will be to host it live on Amazon S3!