Skip to content

Add OpenAI-powered content parsing improvement feature #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

paskal
Copy link
Member

@paskal paskal commented Aug 24, 2024

Resolves #27.

@paskal paskal requested a review from umputun August 24, 2024 10:15
@paskal paskal force-pushed the openai_rules_improver branch from f74825d to 022c3ce Compare October 10, 2024 19:29
@coveralls
Copy link

Pull Request Test Coverage Report for Build 11280720445

Details

  • 66 of 195 (33.85%) changed or added relevant lines in 6 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-12.5%) to 76.539%

Changes Missing Coverage Covered Lines Changed/Added Lines %
backend/extractor/openai_mock.go 13 24 54.17%
backend/rest/server.go 19 49 38.78%
backend/extractor/readability.go 28 116 24.14%
Totals Coverage Status
Change from base Build 11121542707: -12.5%
Covered Lines: 659
Relevant Lines: 861

💛 - Coveralls

@coveralls
Copy link

Pull Request Test Coverage Report for Build 11280720229

Details

  • 66 of 195 (33.85%) changed or added relevant lines in 6 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-12.5%) to 76.539%

Changes Missing Coverage Covered Lines Changed/Added Lines %
backend/extractor/openai_mock.go 13 24 54.17%
backend/rest/server.go 19 49 38.78%
backend/extractor/readability.go 28 116 24.14%
Totals Coverage Status
Change from base Build 11121542707: -12.5%
Covered Lines: 659
Relevant Lines: 861

💛 - Coveralls

@paskal paskal force-pushed the openai_rules_improver branch from 022c3ce to c6ae8bc Compare March 22, 2025 21:08
@paskal paskal force-pushed the openai_rules_improver branch from c6ae8bc to 52d1a71 Compare April 19, 2025 12:44
paskal and others added 3 commits April 21, 2025 23:50
- Introduce 'summary' query parameter in /api/content/v1/parser endpoint
- Integrate OpenAI API for generating article summaries
- Add OpenAIKey field to Server struct and corresponding command-line
flag
- Update extractArticleEmulateReadability to handle summary requests
- Add generateSummary method using OpenAI's GPT-4o model (turns out to
be faster than even 4o mini)
- Add OpenAIClient interface and mock for testing
- Update README.md with new configuration options and API details

This feature allows users to request a summary of extracted articles
using OpenAI's GPT-4o model. To ensure secure usage, summary generation
requires a valid server token. The changes include comprehensive error
handling and test coverage for various scenarios, including token
validation and server misconfiguration.

# Conflicts:
#	backend/go.mod
#	backend/rest/server.go
Implement MongoDB cache for summaries to reduce API costs and improve performance.
Rename parameters from OpenAI-specific to more generic API names.
Support configurable model selection through ModelType enum or direct model names.
Add comprehensive tests with mocks for summary generation and caching.
Update documentation in README with summary feature details.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@paskal paskal force-pushed the openai_rules_improver branch from 52d1a71 to 9e9a926 Compare April 21, 2025 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ChatGPT summary
2 participants