chore: unify robots and fix sitemap freshness, move types to devDeps

[xerion12]

• Remove static sitemap lastmod to avoid stale SEO signals.
• Keep single robots.txt source, remove astro-robots-txt plugin.
• Move all @types/* dependencies to devDependencies to reduce production size.
• Verify builds and update lockfile.

[Fluffy9]

This did build successfully for me although there was a missing "classnames" dependency which may be env related

[HoomanDgtl]

hii @xerion0712 , Thanks for the PR! However, the hardcoded part is intentional. If we make it dynamic, the lastmod value would change with every merged PR — and since we have hundreds of them, it would constantly signal Google to re-crawl pages even when there are no real content or SEO meta changes.

For now, we won’t be able to merge this PR, as it could negatively impact SEO. We’ll discuss this further with Akash’s marketing team and implement the changes based on their recommendations or findings.

We may eventually introduce an updated_at parameter in the frontmatter to ensure the lastmod only updates for pages with actual content changes.

Thanks again for your contribution!

[xerion12]

Thanks for the detail context, it really make sense to me. so I’ll keep the lastmod hardcoded for now like before to prevent google from noisy re-crawls and I will revert the dynamic change in this PR. also i'll leave other parts as is, like robots unification and moving @types packages to devDependencies. for next steps, i wanna try a safer method for lastmod update, only when real content change happen, maybe by using a frontmatter updated_at field in content files so sitemap can use that when it present and skip otherwise. also plan to put some guardrails like limit to content folder only and day level timestamp to avoid tiny updates causing re-crawls. later maybe fallback to git last modified but only for content paths ignoring code changes. if everyone agree, i can prepare a small RFC or doc to marketing team for review before make final PR about updated_at + sitemap transform. also thinking may be some filter by file type/extensions can help too, thank you for every feedback, really appreciate it.

[HoomanDgtl]

sounds good.