aselimov/notex.nvim
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: add relational document system specification and plan

Browse source
This commit is contained in:
Alex Selimov 2025-10-01 22:59:11 -04:00
parent ef65e38bb2
commit 9152f868ce
Signed by: aselimov
GPG key ID: 3DDB9C3E023F1F31
6 changed files with 393 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

30
specs/001-create-a-neovim/data-model.md Normal file
View file

@ -0,0 +1,30 @@
# Data Model: Relational Document System
This document outlines the key entities for the relational document system.
## Entities
### Document
- **Description**: A markdown file with YAML frontmatter.
- **Attributes**:
- `type` (string): The type of the document (e.g., "person", "meeting").
- `properties` (object): A set of key-value pairs defined in the YAML frontmatter.
### Schema
- **Description**: A definition for a document type, specifying its properties and their types. As per the clarification, schemas are not strictly enforced but are inferred from existing documents.
- **Attributes**:
- `name` (string): The name of the document type.
- `properties` (object): A dictionary of property names and their inferred types (e.g., string, number, date).
### Query
- **Description**: A definition in a custom block syntax that filters and sorts documents.
- **Attributes**:
- `document_type` (string): The type of document to query.
- `filters` (array): A list of filter conditions (e.g., `property == value`).
- `sort_by` (array): A list of properties to sort the results by.
### View
- **Description**: A virtual buffer that displays the results of a query.
- **Attributes**:
- `query` (Query): The query that generated the view.
- `documents` (array): A list of documents that match the query.

150
specs/001-create-a-neovim/plan.md Normal file
View file

@ -0,0 +1,150 @@
# Implementation Plan: Relational Document System in Neovim
**Branch**: `001-create-a-neovim` | **Date**: 2025-10-01 | **Spec**: [link to spec.md]
**Input**: Feature specification from `/specs/001-create-a-neovim/spec.md`
## Execution Flow (/plan command scope)
```
1. Load feature spec from Input path
→ If not found: ERROR "No feature spec at {path}"
2. Fill Technical Context (scan for NEEDS CLARIFICATION)
→ Detect Project Type from file system structure or context (web=frontend+backend, mobile=app+api)
→ Set Structure Decision based on project type
3. Fill the Constitution Check section based on the content of the constitution document.
4. Evaluate Constitution Check section below
→ If violations exist: Document in Complexity Tracking
→ If no justification possible: ERROR "Simplify approach first"
→ Update Progress Tracking: Initial Constitution Check
5. Execute Phase 0 → research.md
→ If NEEDS CLARIFICATION remain: ERROR "Resolve unknowns"
6. Execute Phase 1 → contracts, data-model.md, quickstart.md, agent-specific template file (e.g., `CLAUDE.md` for Claude Code, `.github/copilot-instructions.md` for GitHub Copilot, `GEMINI.md` for Gemini CLI, `QWEN.md` for Qwen Code or `AGENTS.md` for opencode).
7. Re-evaluate Constitution Check section
→ If new violations: Refactor design, return to Phase 1
→ Update Progress Tracking: Post-Design Constitution Check
8. Plan Phase 2 → Describe task generation approach (DO NOT create tasks.md)
9. STOP - Ready for /tasks command
```
**IMPORTANT**: The /plan command STOPS at step 7. Phases 2-4 are executed by other commands:
- Phase 2: /tasks command creates tasks.md
- Phase 3-4: Implementation execution (manual or via tools)
## Summary
The project is a Neovim plugin that provides a relational document system. It uses markdown files with YAML frontmatter as documents and supports typed schemas, linking, and query-based views. The queries are defined using a custom block syntax and rendered in virtual buffers. The implementation will be in Lua, leveraging an SQLite database for performance.
## Technical Context
**Language/Version**: Lua (Neovim compatible)
**Primary Dependencies**: lsqlite3 (or similar, to be confirmed by research)
**Storage**: SQLite
**Testing**: busted (or similar, to be confirmed by research)
**Target Platform**: Neovim
**Project Type**: Single project (Neovim plugin)
**Performance Goals**: Query execution should be non-blocking and feel instantaneous for typical workloads.
**Constraints**: Must be implemented in Lua for seamless integration with Neovim.
**Scale/Scope**: The system should be able to handle a library of several thousand documents without significant performance degradation.
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
* **I. Clean Code**: Is the proposed code structure and design clean and maintainable?
* **II. Functional Style**: Does the design favor a functional approach where appropriate?
* **III. Descriptive Coding**: Is the naming of components and files descriptive and self-documenting?
## Project Structure
### Documentation (this feature)
```
specs/001-create-a-neovim/
├── plan.md # This file (/plan command output)
├── research.md # Phase 0 output (/plan command)
├── data-model.md # Phase 1 output (/plan command)
├── quickstart.md # Phase 1 output (/plan command)
└── tasks.md # Phase 2 output (/tasks command - NOT created by /plan)
```
### Source Code (repository root)
```
lua/
└── notex/
├── init.lua
├── schema.lua
├── document.lua
├── query.lua
└── view.lua
tests/
├── spec/
│ ├── schema_spec.lua
│ └── query_spec.lua
└── integration/
└── main_spec.lua
```
**Structure Decision**: A single project structure is chosen, which is standard for Neovim plugins. The core logic will reside in the `lua/notex` directory.
## Phase 0: Outline & Research
1. **Extract unknowns from Technical Context** above:
- Research Lua SQLite libraries (e.g., lsqlite3).
- Research Lua testing frameworks (e.g., busted).
- Research best practices for Neovim plugin development in Lua.
2. **Consolidate findings** in `research.md`.
**Output**: `research.md` with all NEEDS CLARIFICATION resolved.
## Phase 1: Design & Contracts
*Prerequisites: research.md complete*
1. **Extract entities from feature spec** → `data-model.md`.
2. **Extract test scenarios** from user stories → `quickstart.md`.
3. **Update agent file incrementally**.
**Output**: `data-model.md`, `quickstart.md`, and updated agent file.
## Phase 2: Task Planning Approach
*This section describes what the /tasks command will do - DO NOT execute during /plan*
**Task Generation Strategy**:
- Load `.specify/templates/tasks-template.md` as base.
- Generate tasks from Phase 1 design docs (`data-model.md`, `quickstart.md`).
- Each entity in the data model will have corresponding implementation tasks.
- Each scenario in the quickstart guide will have a corresponding integration test task.
**Ordering Strategy**:
- TDD order: Tests before implementation.
- Dependency order: Core data structures and parsing before query and view logic.
- Mark [P] for parallel execution (independent files).
**Estimated Output**: A list of ordered tasks in `tasks.md`.
## Phase 3+: Future Implementation
*These phases are beyond the scope of the /plan command*
**Phase 3**: Task execution (/tasks command creates `tasks.md`)
**Phase 4**: Implementation (execute `tasks.md`)
**Phase 5**: Validation (run tests, execute `quickstart.md`)
## Complexity Tracking
*Fill ONLY if Constitution Check has violations that must be justified*
| Violation | Why Needed | Simpler Alternative Rejected Because |
|-----------|------------|-------------------------------------|
| | | |
## Progress Tracking
*This checklist is updated during execution flow*
**Phase Status**:
- [x] Phase 0: Research complete (/plan command)
- [x] Phase 1: Design complete (/plan command)
- [ ] Phase 2: Task planning complete (/plan command - describe approach only)
- [ ] Phase 3: Tasks generated (/tasks command)
- [ ] Phase 4: Implementation complete
- [ ] Phase 5: Validation passed
**Gate Status**:
- [x] Initial Constitution Check: PASS
- [x] Post-Design Constitution Check: PASS
- [x] All NEEDS CLARIFICATION resolved
- [ ] Complexity deviations documented
---
*Based on Constitution v1.0.0 - See `/memory/constitution.md`*

49
specs/001-create-a-neovim/quickstart.md Normal file
View file

@ -0,0 +1,49 @@
# Quickstart: Relational Document System
This guide provides a quick overview of how to use the relational document system.
## 1. Create a Document
Create a new markdown file (e.g., `people/john-doe.md`) with YAML frontmatter:
```markdown
---
type: person
name: John Doe
email: john.doe@example.com
---
# John Doe
This is a document about John Doe.
```
## 2. Create a Query
Create a new markdown file to define a query (e.g., `views/all-people.md`):
````markdown
```notex
query: person
sort_by: name
```
````
## 3. View the Results
When you open the `views/all-people.md` file, the plugin will execute the query and display the results in a virtual buffer:
```
| name | email |
|----------|------------------------|
| John Doe | john.doe@example.com |
| ... | ... |
```
## 4. Linking Documents
You can link to other documents using the standard markdown syntax:
```markdown
See also: [John Doe](people/john-doe.md)
```

15
specs/001-create-a-neovim/research.md Normal file
View file

@ -0,0 +1,15 @@
# Research: Relational Document System in Neovim
## Research Tasks
### 1. Lua SQLite Library
- **Task**: Research and select a suitable Lua library for interacting with SQLite within a Neovim plugin.
- **Acceptance Criteria**: The library should be well-maintained, performant, and have good documentation.
### 2. Lua Testing Framework
- **Task**: Identify a testing framework for Lua that is compatible with Neovim plugin development.
- **Acceptance Criteria**: The framework should support unit and integration tests, and ideally have a simple setup process.
### 3. Neovim Plugin Development Best Practices
- **Task**: Research best practices for developing Neovim plugins in Lua.
- **Acceptance Criteria**: The research should cover topics like project structure, dependency management, and performance considerations.

96
specs/001-create-a-neovim/spec.md Normal file
View file

@ -0,0 +1,96 @@
# Feature Specification: Relational Document System in Neovim
**Feature Branch**: `001-create-a-neovim`
**Created**: 2025-10-01
**Status**: Draft
**Input**: User description: "Create a neovim plugin representing a relational document system with typed schemas and query-based views. This plugin should support a directory of markdown files each containing yaml metadata representing properties in addition to the "type" of the document. This plugin will support linking between documents as well as creating queries into document properties with support for filtering and sorting by columns. These queries should be rendered in virtual buffers. Queries should be defined using a custom block syntax. This will implement a notion like approach to handling databases and documents."
---
## Clarifications
### Session 2025-10-01
- Q: How are schemas defined and enforced? → A: No strict schema enforcement, only validation of existing properties.
- Q: What should be the syntax for linking between documents? → A: Standard markdown link syntax `[link text](file.md)`.
## ⚡ Quick Guidelines
- ✅ Focus on WHAT users need and WHY
- ❌ Avoid HOW to implement (no tech stack, APIs, code structure)
- 👥 Written for business stakeholders, not developers
### Section Requirements
- **Mandatory sections**: Must be completed for every feature
- **Optional sections**: Include only when relevant to the feature
- When a section doesn't apply, remove it entirely (don't leave as "N/A")
---
## User Scenarios & Testing *(mandatory)*
### Primary User Story
As a Neovim user, I want to manage a personal knowledge base of markdown documents that can be organized and queried like a relational database, so that I can create structured notes and dynamically generate views of my information.
### Acceptance Scenarios
1. **Given** a directory of markdown files with YAML frontmatter, **When** I open a file, **Then** the plugin recognizes the document's type and properties.
2. **Given** a document with a specific type, **When** I define a query in a markdown file to list all documents of that type, **Then** a virtual buffer is created with a table of the matching documents and their properties.
3. **Given** a query with filter and sort conditions, **When** the query is executed, **Then** the results in the virtual buffer are filtered and sorted correctly.
4. **Given** two documents, **When** I create a link from one document to another, **Then** I can navigate between the linked documents.
### Edge Cases
- What happens when a markdown file has invalid YAML frontmatter?
- How does the system handle links to non-existent documents?
- What happens if a query is defined with invalid syntax?
- How does the plugin handle very large numbers of documents or very large individual files?
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: The system MUST recognize a directory of markdown files as a database.
- **FR-002**: The system MUST parse YAML frontmatter in each markdown file to identify the document's type and properties.
- **FR-003**: The system MUST validate document properties against a schema inferred from existing documents of the same type. No strict schema definition file is required.
- **FR-004**: The system MUST support creating links between documents using the standard markdown link syntax (`[link text](file.md)`).
- **FR-005**: The system MUST provide a custom block syntax for defining queries.
- **FR-006**: Queries MUST support filtering by document properties.
- **FR-007**: Queries MUST support sorting by document properties.
- **FR-008**: Query results MUST be displayed in a virtual buffer.
- **FR-009**: The system MUST handle updates to documents and reflect changes in query results.
### Key Entities *(include if feature involves data)*
- **Document**: A markdown file with YAML frontmatter. It has a `type` and a set of `properties`.
- **Schema**: A definition for a document type, specifying its properties and their types.
- **Query**: A definition in a custom block syntax that filters and sorts documents.
- **View**: A virtual buffer that displays the results of a query.
---
## Review & Acceptance Checklist
*GATE: Automated checks run during main() execution*
### Content Quality
- [ ] No implementation details (languages, frameworks, APIs)
- [ ] Focused on user value and business needs
- [ ] Written for non-technical stakeholders
- [ ] All mandatory sections completed
### Requirement Completeness
- [ ] No [NEEDS CLARIFICATION] markers remain
- [ ] Requirements are testable and unambiguous
- [ ] Success criteria are measurable
- [ ] Scope is clearly bounded
- [ ] Dependencies and assumptions identified
---
## Execution Status
*Updated by main() during processing*
- [ ] User description parsed
- [ ] Key concepts extracted
- [ ] Ambiguities marked
- [ ] User scenarios defined
- [ ] Requirements generated
- [ ] Entities identified
- [ ] Review checklist passed
---

53
specs/001-create-a-neovim/tasks.md Normal file
View file

@ -0,0 +1,53 @@
# Tasks: Relational Document System in Neovim
**Input**: Design documents from `/specs/001-create-a-neovim/`
**Prerequisites**: plan.md (required), research.md, data-model.md, quickstart.md
## Format: `[ID] [P?] Description`
- **[P]**: Can run in parallel (different files, no dependencies)
- Include exact file paths in descriptions
## Path Conventions
- Paths shown below assume the project structure defined in `plan.md`.
## Phase 3.1: Setup
- [ ] T001 Create the project structure in `lua/` and `tests/` as defined in `plan.md`.
- [ ] T002 Initialize a Lua project and set up dependencies (lsqlite3, busted).
- [ ] T003 [P] Configure linting and formatting tools for Lua (e.g., luacheck, stylua).
## Phase 3.2: Tests First (TDD) ⚠️ MUST COMPLETE BEFORE 3.3
**CRITICAL: These tests MUST be written and MUST FAIL before ANY implementation**
- [ ] T004 [P] Write a test for schema inference in `tests/spec/schema_spec.lua`.
- [ ] T005 [P] Write a test for document parsing in `tests/spec/document_spec.lua`.
- [ ] T006 [P] Write a test for query parsing in `tests/spec/query_spec.lua`.
- [ ] T007 [P] Write an integration test for creating a document and viewing it in a query in `tests/integration/main_spec.lua`.
## Phase 3.3: Core Implementation (ONLY after tests are failing)
- [ ] T008 Implement schema inference logic in `lua/notex/schema.lua`.
- [ ] T009 Implement document parsing logic in `lua/notex/document.lua`.
- [ ] T010 Implement query parsing logic in `lua/notex/query.lua`.
- [ ] T011 Implement the main plugin logic in `lua/notex/init.lua` to tie everything together.
- [ ] T012 Implement the view logic in `lua/notex/view.lua` to display query results in a virtual buffer.
## Phase 3.4: Integration
- [ ] T013 Integrate the SQLite database for document indexing and querying.
## Phase 3.5: Polish
- [ ] T014 [P] Write unit tests for all public functions.
- [ ] T015 [P] Add documentation for all public functions and configuration options.
- [ ] T016 Perform manual testing based on `quickstart.md`.
## Dependencies
- Tests (T004-T007) before implementation (T008-T012).
- T008, T009, T010 block T011.
- T011 blocks T012.
- T011 blocks T013.
## Parallel Example
```
# Launch T004-T007 together:
Task: "Write a test for schema inference in tests/spec/schema_spec.lua"
Task: "Write a test for document parsing in tests/spec/document_spec.lua"
Task: "Write a test for query parsing in tests/spec/query_spec.lua"
Task: "Write an integration test for creating a document and viewing it in a query in tests/integration/main_spec.lua"
```