Infinite scroll SEO #6598

kontrollanten · 2024-09-20T09:02:25Z

Description

Adds a "Load more" button on the bottom of MyInfiniteScrollerComponent. For a regular user the behavior should be the same.

Behavior for bots:

Loads the page and InifniteScroller will continue to load more content until the visible part of the page is full.
In the bottom the bot will see a link to page?{currentPage++}
Upon visiting this link the scenario will be repeated.

Background

Thoughts during the path

Skip the scroll logic and instead use IntersectionObserver to trigger "near bottom" when the Load more button becomes visible.
I began to add pagination URL query params upon scroll, but this went too complex. Maybe it can be nice for users to be able to link directly to a paginated page, but it's out of scope for this feature.
When watching a video in a playlist the query params will affect both the playlist and comments. Not ideal, but will only affect bots.

Todo

Add page query param to the canonical URL for pages where the main content is paginated. For other paginated pages (i.e. comments, watch playlist, etc) the canonical URL shouldn't include the page query param.

Related issues

#6332

Has this been tested?

👍 yes, I added tests to the test suite
💭 no, because this PR is a draft and still needs work
🙅 no, because this PR does not update server code
🙋 no, because I need help

To test it manually, add a "?page=1" query param to a route which has infinite scroll

Screenshots

client/src/app/+my-library/+my-video-channels/my-video-channels.component.ts

client/src/app/+my-library/my-follows/my-followers.component.ts

client/src/app/+my-library/my-follows/my-subscriptions.component.ts

client/src/app/+my-library/my-video-playlists/my-video-playlists.component.ts

client/src/app/+search/search.component.ts

client/src/app/shared/shared-video-miniature/videos-list.component.ts

client/src/app/shared/shared-video-miniature/videos-selection.component.ts

client/src/app/shared/standalone-notifications/user-notifications.component.ts

server/core/controllers/index.html

server/core/lib/html/shared/video-html.ts

kontrollanten · 2024-09-24T18:44:04Z

@Chocobozzz I did a self-review and found some minor style changes. If you're okay with this implementation I'll fix them.

kontrollanten · 2024-10-21T08:49:14Z

@Chocobozzz Any thoughts on this? If this feature is wanted I'll do a cleanup and fix merge conflicts.

Chocobozzz · 2024-10-21T09:21:13Z

Any thoughts on this? If this feature is wanted I'll do a cleanup and fix merge conflicts.

We're interested in the feature but unfortunately we don't have time to review it yet

Add a "Load more" button in the bottom to help search engine bots to navigate to the next page. In order to debug this functionality, add ?finiteScroll=true to the URL.

hasMoreItems has to be called before bumping the current page in order to work accordingly.

closes Chocobozzz#6332

Chocobozzz · 2025-09-10T12:05:18Z

Hi and sorry for the (very) late review.
Unfortunately this PR complicates a lot the infinite scroll behaviour. Can you confirm that you stil have SEO issues, even with the sitemap provided by PeerTube?

If so, can we implement instead a logic for robots where we inject a rel=next/rel=prev links in the header with page=... query param, and update the component to set currentPage = query.page in ngOnInit? With this method we minimize changes.

kontrollanten · 2025-10-06T02:21:08Z

Can you confirm that you stil have SEO issues, even with the sitemap provided by PeerTube?

Yes, the issue persists and according to #6210 (comment) we doesn't seem to be alone.

If so, can we implement instead a logic for robots where we inject a rel=next/rel=prev links in the header with page=... query param, and update the component to set currentPage = query.page in ngOnInit? With this method we minimize changes.

We can always try, but it's not according to Googles recommendations.

A simpler solution would be to create separate logic for the /videos/browse only and use IntersectionObserver to handle the loading. That would reduce the complexity a lot.

Chocobozzz · 2025-10-08T13:13:12Z

We can always try, but it's not according to Googles recommendations.

Nice catch.

A simpler solution would be to create separate logic for the /videos/browse only and use IntersectionObserver to handle the loading. That would reduce the complexity a lot.

What if we mix the solutions:

Still use the infinite scroll directive
Add a "Load more" button in Browse videos page only that would link to the next page
Update the Browse videos page to init set currentPage = query.page in ngOnInit

?

And how would you use IntersectionObserver in your proposal? Would it observe the "Load more" button to know when to load more results?

kontrollanten · 2025-10-13T11:50:06Z

What if we mix the solutions:

Still use the infinite scroll directive

Add a "Load more" button in Browse videos page only that would link to the next page

Update the Browse videos page to init set currentPage = query.page in ngOnInit

What would be the difference from this PR:s solution? I think it would be simpler create a new directive only used in scenarios where SEO is needed. A one size fits all solution was hard to get clean, at least one year ago.

And how would you use IntersectionObserver in your proposal? Would it observe the "Load more" button to know when to load more results?

Yes, and to to know when to update the URL.

Google also recommends to:

Link sequentially to the individual URLs so that search engines can discover the URLs in a paginated set. Find out more about best practices when implementing pagination.

So the best scenario would be to have a pagination navigation, rather than a simple "Load more" button.

Chocobozzz · 2025-11-13T13:27:24Z

(sorry for my late answer)

What would be the difference from this PR:s solution? I think it would be simpler create a new directive only used in scenarios where SEO is needed. A one size fits all solution was hard to get clean, at least one year ago.

The difference is that we add the "Load more" button directly in the Browse Videos component, not inside the infinite scroll directive. With the proposed solution, we don't change how we load videos and still use the infinite scroll directive. It seems to me this solution is easier to add.

Anyway this is just implementation details, I think we agree on the global solution. If you want to give it a try, don't hesitate to force push on this PR or create another one.

kontrollanten marked this pull request as draft September 20, 2024 19:04

kontrollanten force-pushed the feat-6332-infinite-scroll-seo-2 branch 2 times, most recently from ad1d2d6 to b620bea Compare September 24, 2024 10:12

kontrollanten marked this pull request as ready for review September 24, 2024 10:44

kontrollanten commented Sep 24, 2024

View reviewed changes

kontrollanten force-pushed the feat-6332-infinite-scroll-seo-2 branch 2 times, most recently from 812fa93 to b9dadbf Compare October 23, 2024 11:22

kontrollanten added 3 commits October 23, 2024 13:22

chore: fix eslint for vs code

933ec9c

feat(client/videos-list): infinite scroll SEO

7b9adce

Add a "Load more" button in the bottom to help search engine bots to navigate to the next page. In order to debug this functionality, add ?finiteScroll=true to the URL.

fix(client/user-notifications): load last page

52aba7a

hasMoreItems has to be called before bumping the current page in order to work accordingly.

kontrollanten force-pushed the feat-6332-infinite-scroll-seo-2 branch from b9dadbf to 1ac7373 Compare October 23, 2024 11:42

feat(client/infinite-scroll): SEO friendly

a91fd0c

closes Chocobozzz#6332

kontrollanten force-pushed the feat-6332-infinite-scroll-seo-2 branch 2 times, most recently from c7a1037 to 66db1ea Compare October 23, 2024 12:41

feat: add canonical tag to pages with pagination

1066288

kontrollanten force-pushed the feat-6332-infinite-scroll-seo-2 branch from 66db1ea to 1066288 Compare October 24, 2024 04:26

Chocobozzz added the Status: Waiting for maintainer label Jan 31, 2025

Chocobozzz added Status: Waiting for answer Waiting issue author answer and removed Status: Waiting for maintainer labels Sep 10, 2025

Uh oh!

Infinite scroll SEO #6598

Are you sure you want to change the base?

Infinite scroll SEO #6598

Conversation

kontrollanten commented Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Background

Thoughts during the path

Todo

Related issues

Has this been tested?

Screenshots

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kontrollanten commented Sep 24, 2024

Uh oh!

kontrollanten commented Oct 21, 2024

Uh oh!

Chocobozzz commented Oct 21, 2024

Uh oh!

Chocobozzz commented Sep 10, 2025

Uh oh!

kontrollanten commented Oct 6, 2025

Uh oh!

Chocobozzz commented Oct 8, 2025

Uh oh!

kontrollanten commented Oct 13, 2025

Uh oh!

Chocobozzz commented Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kontrollanten commented Sep 20, 2024 •

edited

Loading