# Article Token Search (Advanced) ## Example Overview The **Article Token Search (Advanced)** example is the companion to the {doc}`basic Article Token-Style Search `. Both demos use `TomSelectTokenWidget` against a `CompositeAutocompleteView`; the difference is which extension point of `Operator` each one exercises. - The basic demo uses `Operator.filter_lookup` — exact / `__in` matching only. - This advanced demo uses `Operator.q_translator` — a callable that receives `(op, list[values])` and returns an arbitrary `Q` object, unlocking date comparisons, numeric ranges, and any other ORM lookup you can express in `Q`. **Objective**: - Demonstrate `Operator.q_translator` as the path for non-equality lookups. - Show how to encode comparison and range syntax inside the token *value*, since the tokenizer only understands `key:value` (and comma-multi). - Show clean error surfacing when the user types an invalid value (`published_after:not-a-date`, `word_count:abc`). **Use Case**: - Triage / inbox / log-search UIs that need date and numeric filters in addition to equality filters. - Bookmark-driven saved views combining "documents modified after X with size between A and B written by author Y" in a single URL. **Visual Examples** ![Screenshot: Advanced token search — date and range operators](https://raw.githubusercontent.com/OmenApps/django-tomselect/refs/heads/main/docs/images/article-advanced-token-search-published.png) ![Screenshot: Advanced token search — word count](https://raw.githubusercontent.com/OmenApps/django-tomselect/refs/heads/main/docs/images/article-advanced-token-search-word-count.png) ![Screenshot: Advanced token search — filtered results](https://raw.githubusercontent.com/OmenApps/django-tomselect/refs/heads/main/docs/images/article-advanced-token-search.png) --- ## Key Code Segments ### `q_translator` callables The translator receives the `Operator` and the parsed list of token values (length 1 for `multi=False`, length N for `multi=True`). It must return a `Q` object. Raising `ValueError` is the supported way to signal a bad value — the parser wraps it into a `ValidationError` for the form. ```python from django.db.models import Q from django.utils.dateparse import parse_date def _parse_iso_date(values): raw = (values[0] or "").strip() parsed = parse_date(raw) if not parsed: raise ValueError(f"Invalid date: {raw!r}. Use YYYY-MM-DD.") return parsed def _q_published_after(op, values): return Q(created_at__date__gte=_parse_iso_date(values)) def _q_word_count(op, values): """Accepts >500, <2000, >=1000, <=5000, =500, 100..2000, or a plain int.""" raw = (values[0] or "").strip() if ".." in raw: lo, hi = raw.split("..", 1) return Q(word_count__gte=int(lo), word_count__lte=int(hi)) for prefix, lookup in ((">=", "gte"), ("<=", "lte"), (">", "gt"), ("<", "lt"), ("=", "exact")): if raw.startswith(prefix): return Q(**{f"word_count__{lookup}": int(raw[len(prefix):])}) return Q(word_count__exact=int(raw)) ``` ### `NoSuggestionAutocompleteView` `Operator` requires `view` to be set even when `q_translator` handles the filtering, because the widget can also call `?mode=value&op=&q=...` to populate a value dropdown. For free-form operators (dates, comparisons) there *is* nothing useful to suggest, so we bind those operators to a stub view that returns an empty queryset. Without this stub, typing `published_after:` would pop unrelated article suggestions whose ids would then be the wrong type for `_q_published_after`. ```python class NoSuggestionAutocompleteView(AutocompleteModelView): """Empty-result placeholder for operators with free-form values.""" model = Article value_fields = ["id"] skip_authorization = True def get_queryset(self): return Article.objects.none() ``` ### Composite view with mixed operator kinds ```python class ArticleAdvancedTokenQueryView(CompositeAutocompleteView): operators = [ Operator(key="author", view=AuthorAutocompleteView, value_field="id", label_field="name", filter_lookup="authors__id", multi=True), Operator(key="status", view=ArticleStatusAutocompleteView, value_field="value", label_field="label", filter_lookup="status", multi=True), Operator(key="published_after", view=NoSuggestionAutocompleteView, value_field="id", label_field="id", q_translator=_q_published_after, max_count=1), Operator(key="published_before", view=NoSuggestionAutocompleteView, value_field="id", label_field="id", q_translator=_q_published_before, max_count=1), Operator(key="word_count", view=NoSuggestionAutocompleteView, value_field="id", label_field="id", q_translator=_q_word_count, max_count=1), ] free_text_lookups = ["title__icontains"] ``` ### Form ```python from django_tomselect.forms import TomSelectTokenField class ArticleAdvancedTokenSearchForm(forms.Form): q = TomSelectTokenField( composite_view="autocomplete-article-advanced-token", required=False, allow_free_text=True, max_tokens=20, ) ``` ### View Error handling matches the basic token demo. `ParsedQuery.apply()` catches `ValueError` and `TypeError` raised by `q_translator` callables and re-raises them as `ValidationError`, which the view surfaces via `form.add_error("q", exc)`. ```python from django.core.exceptions import ValidationError from django_tomselect.query import parse_query def article_advanced_token_search_view(request): form = ArticleAdvancedTokenSearchForm(request.GET or None) articles = Article.objects.none() if form.is_valid(): q = form.cleaned_data.get("q", "") or "" if q: parsed = parse_query(q, ArticleAdvancedTokenQueryView) try: articles = parsed.apply(Article.objects.all()).distinct()[:50] except ValidationError as exc: form.add_error("q", exc) return TemplateResponse( request, "example/advanced_demos/article_advanced_token_search.html", {"form": form, "articles": articles}, ) ``` ### URL Wiring ```python path( "autocomplete/article-advanced-token/", autocompletes.ArticleAdvancedTokenQueryView.as_view(), name="autocomplete-article-advanced-token", ), path( "autocomplete/no-suggestion/", autocompletes.NoSuggestionAutocompleteView.as_view(), name="autocomplete-no-suggestion", ), path( "article-advanced-token-search/", views.article_advanced_token_search_view, name="article-advanced-token-search", ), ``` --- ## Try It | Action | Result | |---|---| | Type `published_after:2024-01-01` | Articles created on or after 2024-01-01. | | Type `published_before:2024-12-31` | Articles created strictly before 2024-12-31. | | Combine `published_after:... published_before:...` | Inclusive-start, exclusive-end date range. | | Type `word_count:>500` | Articles longer than 500 words. | | Type `word_count:100..2000` | Inclusive numeric range. | | Type `word_count:>=1000` | Greater-or-equal comparison. | | Type `status:draft,published` | Multi-value via comma (`filter_lookup` path, unchanged). | | Type `published_after:not-a-date` | Inline `ValidationError`: "Invalid date: 'not-a-date'. Use YYYY-MM-DD." | | Type `published_after:` (no value) | Tom Select dropdown shows nothing (the placeholder view returns an empty queryset). | | Bookmark a URL with a tokenized query | Chips rehydrate on reload, including the free-form date/range tokens. | --- ## When to choose `q_translator` vs. `filter_lookup` - **`filter_lookup`**: when the user picks a value from a dropdown that maps to an ORM `__in` / equality lookup. The bound view's autocomplete suggests valid values; the parsed token value is one of those ids. - **`q_translator`**: when the value cannot be enumerated (dates, numbers, free-form expressions) or when you need a non-trivial lookup (range, geo-distance, full-text rank, JSON path, embedding similarity). Pair with a `NoSuggestionAutocompleteView` if browsing the bound view's data is not a useful UX for that operator. --- ## Related - {doc}`article_token_search` — the basic version that uses only `filter_lookup`. - API reference: `TomSelectTokenWidget`, `TomSelectTokenField`, `CompositeAutocompleteView`, `Operator`, `parse_query`.