Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
e34f8a5
Integración de ArticleViewSet en el enrutador y actualización de URLs…
eduranm Sep 26, 2025
ab31a5e
Actualización de settings: cambio en directorio de modelos, inclusión…
eduranm Sep 26, 2025
b2e9538
Ampliación de core.models: nuevos modelos de género, idioma, licencia…
eduranm Sep 26, 2025
d7395d9
Actualización en Reference: uso de ReferenceStatus y ajuste en creaci…
eduranm Sep 26, 2025
02654a5
Refactor en reference.config: serialización de ejemplos con json.dump…
eduranm Sep 26, 2025
e40b6be
Refactor en marker: actualización de importación de GenericLlama desd…
eduranm Sep 26, 2025
d88c9c5
Eliminación de reference.tasks (responsabilidad movida a data_utils y…
eduranm Sep 26, 2025
3159c3b
Refactor en wagtail_hooks: uso de SnippetViewSet y soporte para desca…
eduranm Sep 26, 2025
8746d02
Actualización de dependencias: inclusión de langid, google-generative…
eduranm Sep 26, 2025
a9305e6
Creación y ampliación de la app core: modelos comunes, formularios, v…
eduranm Sep 26, 2025
95a8153
Nueva app markup_doc: modelos ArticleDocx, API REST, tareas de marcad…
eduranm Sep 26, 2025
58cc438
Nueva librería markuplib: funciones para procesar DOCX y transformaci…
eduranm Sep 26, 2025
4944e3b
Nueva app model_ai: integración de LLaMA, funciones genéricas, mensaj…
eduranm Sep 26, 2025
4ecb50a
Módulos adicionales en reference: configuración de Gemini y utilidade…
eduranm Sep 26, 2025
2ab9bd9
Consolidación de migraciones en reference: actualización de 0001_init…
eduranm Sep 26, 2025
3024568
Actualización de migración inicial en core_settings: cambio de versió…
eduranm Sep 26, 2025
1e31de7
Creación de migración inicial en core
eduranm Sep 26, 2025
826800b
Eliminación de la app llama3: funcionalidades migradas a model_ai
eduranm Sep 26, 2025
76258cd
fix(markup_doc): capturar ArticleDocxMarkup.DoesNotExist en generate_xml
eduranm Oct 5, 2025
aff3a39
refactor(markuplib): eliminar prints de depuración en extracción de e…
eduranm Oct 5, 2025
6dd1ae2
style(model_ai): reemplazar bare except por Exception en download_model
eduranm Oct 5, 2025
5842054
fix(reference): corregir literales y separadores en config_gemini par…
eduranm Oct 5, 2025
6a09911
i18n(reference): envolver labels y help_text con gettext_lazy
eduranm Oct 5, 2025
95e6d31
model_ai: corrige etiqueta de i18n y refuerza unicidad de LlamaModel …
eduranm Oct 17, 2025
192403d
reference: renombra campo 'estatus' a 'status' y ajusta etiqueta a 'R…
eduranm Oct 17, 2025
22fe3d1
reference: elimina import no utilizado en wagtail_hooks.py
eduranm Oct 17, 2025
abe54e2
remove antigo app llama3 e resolve conflito em base.py
pitangainnovare Oct 20, 2025
e000258
adiciona instruções para fazer build com suporte a Llama
pitangainnovare Oct 20, 2025
cbc8509
melhora imports
pitangainnovare Oct 20, 2025
2c32138
Apaga modulo legado llama3
pitangainnovare Oct 20, 2025
aa9c6a3
Resolve conflitos em generic_llama e aplica correções estruturais:
pitangainnovare Oct 20, 2025
dc61b24
Cria input settings para references
pitangainnovare Oct 20, 2025
e2e696a
Padroniza deps
pitangainnovare Oct 20, 2025
3e712af
Padroniza imports em geral (além de adequar à nova nomenclatura)
pitangainnovare Oct 20, 2025
dd94a35
Adiciona método faltante em marker (por causa de merge)
pitangainnovare Oct 20, 2025
3ed8e58
Remove linhas comentadas
pitangainnovare Oct 20, 2025
1fbd7d1
Inclui AI Model na interace, para poder registrar modelos
pitangainnovare Oct 20, 2025
e22bc7c
Adequa uso do serviço Llama em tasks
pitangainnovare Oct 20, 2025
79245bd
Adiciona migrações
pitangainnovare Oct 20, 2025
5740120
Corrige nome de parâmetro type (deve ser mode)
pitangainnovare Oct 20, 2025
a830fb5
Flexibiliza attrs de LlamaService
pitangainnovare Oct 20, 2025
3bf5c7c
Adiciona alguns FIXME
pitangainnovare Oct 20, 2025
f501379
Melhora imports
pitangainnovare Oct 20, 2025
cb739b4
Padroniza nome de metodo que obtem tipo de IA (LLAMA ou GEMINI)
pitangainnovare Oct 20, 2025
08f465e
Corrige o nome do site
Rossi-Luciano Oct 28, 2025
bb461e0
Atualiza os conteúdos de 'locale'
Rossi-Luciano Oct 28, 2025
4fa1b11
Correções em 'wagtail_hooks'
Rossi-Luciano Oct 28, 2025
acb914a
Corrige atributo
Rossi-Luciano Oct 28, 2025
f218c87
Corrige 'data_utils'
Rossi-Luciano Oct 28, 2025
e32247d
refactor: reorganiza estrutura de volumes Docker
Rossi-Luciano Nov 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,4 @@ wheels/
*.egg-info/
.installed.cfg
*.egg
.ipython/
18 changes: 17 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -173,4 +173,20 @@ cython_debug/

# End of https://www.toptal.com/developers/gitignore/api/django

llama3/llama-3.2/*
llama3/llama-3.2/*
# Backups locais
*.backup
Makefile.backup
core/media.backup/

# Arquivos de trabalho temporário
diff.txt

# Modelos de AI (muito grandes)
models/

# Binários
appimagetool

# Migrations geradas localmente
reference/migrations/0002_rename_estatus_to_status.py
9 changes: 4 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ stop: ## Stop all app using $(compose)

restart:
@docker compose -f $(compose) restart

ps: ## See all containers using $(compose)
@docker compose -f $(compose) ps

Expand Down Expand Up @@ -102,10 +102,10 @@ django_load_auth: ## Run manage.py dumpdata auth --indent=2 $(compose)
@docker compose -f $(compose) run --rm django python manage.py loaddata --database=default fixtures/auth.json

dump_data: ## Dump database into .sql $(compose)
docker exec -t scielo_markup_local_postgres pg_dumpall -c -U debug > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql
docker exec -t markapi_local_postgres pg_dumpall -c -U debug > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql

restore_data: ## Restore database into from latest.sql file $(compose)
cat backup/latest.sql | docker exec -i scielo_markup_local_postgres psql -U debug
cat backup/latest.sql | docker exec -i markapi_local_postgres psql -U debug

############################################
## Atalhos Úteis ##
Expand Down Expand Up @@ -133,5 +133,4 @@ clean_migrations: ## Remove all migrations
@echo "Migrations cleaned successfully."

clean_celery_logs:
@sudo truncate -s 0 $$(docker inspect --format='{{.LogPath}}' scielo_markup_local_celeryworker)

@sudo truncate -s 0 $$(docker inspect --format='{{.LogPath}}' markapi_local_celeryworker)
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Targets:
vcs_ref Show last commit ref
build_date Show build date
build Build app using $(COMPOSE_FILE_DEV)
build_llama Build app using llama.local.yml
up Start app using $(COMPOSE_FILE_DEV)
logs Show logs using $(COMPOSE_FILE_DEV)
stop Stop app using $(COMPOSE_FILE_DEV)
Expand All @@ -58,14 +59,19 @@ Targets:

### Common Commands

Build the development environment:
Build the development environment without Llama support:

```bash
make build compose=local.yml
# or simply
make
```

Build the development environment with Llama support:
```bash
make build compose=llama.local.yml
```

Start the project:

```bash
Expand Down
4 changes: 3 additions & 1 deletion config/api_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@
from rest_framework.routers import DefaultRouter, SimpleRouter

from reference.api.v1.views import ReferenceViewSet
from markup_doc.api.v1.views import ArticleViewSet

app_name = "reference"
#app_name = "reference"

if settings.DEBUG:
router = DefaultRouter()
else:
router = SimpleRouter()

router.register("reference", ReferenceViewSet, basename="reference")
router.register("first_block", ArticleViewSet, basename="first_block")

urlpatterns = router.urls
12 changes: 9 additions & 3 deletions config/settings/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
ROOT_DIR = Path(__file__).resolve(strict=True).parent.parent.parent
# core/
APPS_DIR = ROOT_DIR / "core"
LLAMA_MODEL_DIR = ROOT_DIR / "model_ai/download"

env = environ.Env()
READ_DOT_ENV_FILE = env.bool("DJANGO_READ_DOT_ENV_FILE", default=False)
Expand Down Expand Up @@ -80,6 +81,9 @@
"tracker",
"reference",
"xml_manager",
"markup_doc",
"markuplib",
"model_ai"
]

INSTALLED_APPS = DJANGO_APPS + THIRD_PARTY_APPS + LOCAL_APPS + WAGTAIL
Expand Down Expand Up @@ -214,7 +218,7 @@

# Wagtail settings

WAGTAIL_SITE_NAME = "markapi"
WAGTAIL_SITE_NAME = "SciELO XML Tools"

# Search
# https://docs.wagtail.org/en/stable/topics/search/backends.html
Expand Down Expand Up @@ -296,5 +300,7 @@

# LLAMA
LLAMA_ENABLED = env.bool("LLAMA_ENABLED", default=False)
LLAMA_MODEL_DIR = ROOT_DIR / "llama3/llama-3.2"
MODEL_LLAMA = "llama-3.2-3b-instruct-q4_k_m.gguf"
MODEL_LLAMA = "llama-3.2-3b-instruct-q4_k_m.gguf"

#Aumento en el límite de campos
DATA_UPLOAD_MAX_NUMBER_FIELDS = 10000
7 changes: 5 additions & 2 deletions config/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,26 @@

from core.search import views as search_views
from reference import views as reference_views
from config import api_router as api_router

urlpatterns = [
path("admin/autocomplete/", include(autocomplete_admin_urls)),
#path("admin/autocomplete/", include(autocomplete_admin_urls)),
path("django-admin/", admin.site.urls),
path("admin/", include(wagtailadmin_urls)),
path("documents/", include(wagtaildocs_urls)),
path("search/", search_views.search, name="search"),
# JWT
path("api/v1/auth/token/", TokenObtainPairView.as_view(), name="token_obtain_pair"),
path("api/v1/auth/token/refresh/", TokenRefreshView.as_view(), name="token_refresh"),
path("api/v1/mix_citation/", include("config.api_router", namespace="reference")),
path("api/v1/", include(api_router)),
#path("api/v1/mix_citation/", include("config.api_router", namespace="reference")),
# URL para trocar idioma
path('i18n/', include('django.conf.urls.i18n')),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

# URLs com prefixo de idioma
urlpatterns += i18n_patterns(
path("admin/autocomplete/", include(autocomplete_admin_urls)),
path("admin/", include(wagtailadmin_urls)),
path("documents/", include(wagtaildocs_urls)),
path("search/", search_views.search, name="search"),
Expand Down
Loading