No More Surprises!

With experience in both development and SEO consultancy, I've encountered a variety of SEO challenges - some rewarding, others less so. My goal is to minimise these less favourable surprises in my work. By blending my expertise in SEO and development, I aim to offer insights into the solutions I've devised, ranging from highly effective to moderately successful, to help bridge the gap between SEO and development

Where it all started

Once upon a time, in the fast-paced realm of e-commerce, I found myself at the forefront of a significant website enhancement. Months of meticulous work, including adjustments to the payment provider, culminated in a seemingly flawless implementation. The acceptance environment sparkled with perfection, and we were comfortably ahead of our deadline. The much-anticipated live day arrived, and as jubilation filled the air, a quick website crawl revealed a chilling truth - the entire site was set to 'no-index.' Amidst the celebration, I couldn't help but berate myself for overlooking such a critical detail.

In the aftermath of this unexpected setback, I was haunted by a pressing question. How could we ensure that surprises like these were prevented in the future, even before a single line of code went live? The landscape of SEO tools, although formidable, proved to be more reactive than proactive. Tools like ScreamingFrog and Contentking, while excellent in catching errors post-deployment, left a gap in safeguarding against unforeseen issues before going live. Time, as we all know, is money, and every second of a page's invisibility amounts to a potential loss.

Venturing into the realm of Continuous Integration and Continuous Deployment (CICD), I discovered the challenges faced by SEO specialists. Tracking the myriad changes in code within such an environment was akin to chasing a constantly shifting target. The consequence? Unseen alterations that could have a significant impact on a website's visibility. Picture a scenario where a page, unbeknownst to anyone, adopts a 'no-index' stance or experiences a sudden change in its title. These seemingly minor alterations could wreak havoc on page rankings and subsequently affect organic traffic.

In my case, the culprit was a modified robots.txt file during the deployment to the production environment, resulting in the entire domain being tagged as 'no-index.' Fueled by this experience, a burning question lingered in my mind for a long time: What if we could eliminate surprises altogether? What if, before any code hit the live servers, we could catch these potential SEO pitfalls?

Enter the realm of automated end-to-end tests

These tests, designed to simulate how a user or search engine would interact with specific pages on the live site, can be our proactive saviours. By integrating these tests into the CICD pipeline, the deployment process becomes a litmus test for potential SEO issues. If any test fails, signalling a deviation from the expected outcome, the entire deployment is paused. It is only allowed to proceed once the underlying code is fixed and the test successfully passes. No more surprises after the fact - a revolution in the way you can safeguard your website's SEO integrity.

SEO, as we know, thrives on details. Elements like canonical link elements, robots meta-information, page titles, descriptions, and href-language tags play a crucial role in a website's indexation journey. Recognising this, you can craft specific tests for each of these elements. These tests, which will become an integral part of the deployment process, automatically scrutinised the (live) environment with each code update. A seamless deployment means that all SEO-centric tests pass, providing assurance that no inadvertent changes jeopardised the vital components of our website.

Setting up end-to-end tests for SEO

Now, let's shift our focus to the meticulous process of SEO testing, an essential part of the development flow. Encoded within the following Cypress code block are various tests meticulously crafted for SEO validation. Leveraging Cypress, a JavaScript end-to-end test framework, we scrutinize our homepage to ensure its compliance with crucial SEO elements.

describe('SEO tests for the homepage', () => {
 beforeEach(() => {
  cy.visit('/')
 })


it('should have the robots metatag set to index follow', () => {
 cy.get('head meta[name=robots]').should(
  'have.attr',
  'content',
  'index,follow',
 );
})


it('should have a meta description', () => {
 cy.get('head meta[name=description]').should(
  'have.attr',
  'content').and('contain', 'trust me I`m a bot - beep boop');
})


it('should have a page title', () => {
 cy.get('head title').should(
  'include.text', 'Creating tests for the purpose of SEO')
})


 it('should have links to the same page in a different languages', () => {
  cy.get('head link[rel=alternate]').should('have.attr','href','/fr')
 })
})

1. Test Suite Description:

describe('SEO tests for the homepage', () => {

This describes a test suite titled "SEO tests for the homepage" using the describe function. Within this suite, there are multiple individual tests specified using the it function.

1. Before Each Test:

beforeEach(() => {
cy.visit('/')
})

The beforeEach hook runs before each test in the suite. In this case, it ensures that the webpage is visited (cy.visit('/')) before each test is executed. The '/' represents the homepage URL.

1. Test 1 - Checking Robots Meta Tag:

it('should have the robots metatag set to index follow', () => {
cy.get('head meta[name=robots]').should(
'have.attr',
'content',
'index,follow',
)})

This test checks whether the meta tag with the name 'robots' in the <head> section of the page has its 'content' attribute set to 'index,follow'.

1. Test 2 - Checking Meta Description:

it('should have a meta description', () => {
cy.get('head meta[name=description]').should(
'have.attr',
'content').and('contain', 'trust me I`m a bot - beep boop')
})

This test verifies the presence of a meta tag with the name 'description' in the <head> section. It also checks if the 'content' attribute contains the specified text.

1. Test 3 - Checking Page Title:

it('should have a page title', () => {
cy.get('head title').should(
'include.text', 'Creating tests for the purpose of SEO')
})

This test ensures that the <head> section contains a title tag, and the text of the title includes the specified string.

1. Test 4 - Checking Links to Different Languages:

it('should have links to the same page in a different languages', () => {
 cy.get('head link[rel=alternate]').should('have.attr','href','/fr')
 })
})

This test checks for the presence of link tags with the attribute 'rel' set to 'alternate' in the <head> section. It also verifies that the 'href' attribute is set to '/fr', suggesting a link to the French version of the page.

The tests above should give you an idea of how to set them up and what bits you can test. You can extend them with for example a test for a canonical link element, or the <h1>-header only being present once.

Saving time and resources

But it's not just about executing tests; it's about creating a system of alerts tailored to specific scenarios. In the premium version of Cypress, alerts can be customised based on tags. These tags may include version numbers or deployment environments, linking directly to designated Slack channels. For instance, if tests tagged as "production" fail, a notification could be sent to a channel where on-call developers reside, ensuring swift resolution if a deployment issue arises.

Moving beyond mere code execution, this proactive approach extends into the realm of development workflow integration. As an advocate for SEO's involvement in the development process, I propose a paradigm shift. SEO specialists, much like Quality Assurance (QA) counterparts, could be assigned tickets when there's a suspicion that a task might impact the website's SEO. Given the intricate connection between website technology and SEO, almost every ticket becomes worth a thorough SEO check. This approach not only provides SEO specialists with a better understanding of development processes but also opens avenues for collaborative refinement processes.

However, adopting this holistic approach to SEO does introduce additional steps into the process, potentially elongating time-to-market. Yet, as we weigh the cost of time against the consequences of invisibility on Google, the investment proves worthwhile. A meticulous approach to SEO during development not only safeguards against post-deployment issues but also sets the stage for efficient, error-free coding. After all, avoiding the need to redo tasks due to SEO misalignment ultimately saves both time and resources.

Lastly, we must acknowledge certain considerations when crafting end-to-end tests. These tests, designed to evaluate a specific page, pose challenges in maintaining consistency across local, acceptance, and production environments. While a test should ideally be content agnostic, ensuring its applicability in diverse settings, the dynamic nature of content introduces complexities. This leads us to favour tests that focus on structural and logical elements rather than content specifics. While technical requirements such as ensuring the homepage is not set to 'no-index' can be straightforward, validating content-specific details may pose challenges. In such cases, the test's responsibility lies in verifying technical prerequisites, leaving content nuances to the discernment of SEO specialists.

And so,

the story concludes with a transformed approach to website deployments. No longer are we at the mercy of post-deployment surprises, but rather architects of a process that ensures the seamless integration of new features without compromising our SEO foundation. With automated tests standing guard, our journey through the ever-evolving landscape of SEO is marked by confidence, foresight, and the ability to script our own narrative.

If you have any questions regarding this and the implementation feel free to reach out. Thanks for reading.