notex.nvim/README.md

382 lines
8.6 KiB
Markdown

# Notex.nvim
A relational document system for Neovim that brings Notion-like database capabilities to your markdown files. Vibe-coded with z.ai glm-4.6.
![Notex.nvim](https://img.shields.io/badge/Neovim-0.7+-green.svg)
![License](https://img.shields.io/badge/License-MIT-blue.svg)
## Features
- **Relational Document Management**: Index markdown files and query them like databases
- **Custom Query Syntax**: Simple, powerful query language for document discovery
- **Virtual Buffers**: Interactive query results displayed in Neovim buffers
- **YAML Header Parsing**: Automatically extract and index document metadata
- **Real-time Updates**: Automatic reindexing when files change
- **Performance Optimized**: Multi-tier caching system for fast queries
- **Extensible**: Plugin architecture for custom parsers and views
## Quick Start
### Installation
Using [packer.nvim](https://github.com/wbthomason/packer.nvim):
```lua
use {
'your-username/notex.nvim',
requires = {
'nvim-lua/plenary.nvim',
'nvim-tree/nvim-web-devicons'
},
config = function()
require('notex').setup({
-- Configuration options
database_path = vim.fn.stdpath('data') .. '/notex/notex.db',
auto_index = true,
performance = {
enable_caching = true,
cache_size = 100
}
})
end
}
```
Using [vim-plug](https://github.com/junegunn/vim-plug):
```vim
Plug 'your-username/notex.nvim'
Plug 'nvim-lua/plenary.nvim'
Plug 'nvim-tree/nvim-web-devicons'
lua << EOF
require('notex').setup({
database_path = vim.fn.stdpath('data') .. '/notex/notex.db',
auto_index = true
})
EOF
```
### Basic Usage
1. **Index your workspace**:
```vim
:lua require('notex').index_workspace()
```
2. **Run a query**:
```vim
:lua require('notex').show_query_prompt()
```
3. **Example queries**:
```
FROM documents WHERE status = "published"
FROM documents WHERE tags LIKE "project" ORDER BY updated_at DESC
FROM documents WHERE created_at > "2023-01-01" LIMIT 20
```
## Configuration
```lua
require('notex').setup({
-- Database configuration
database_path = nil, -- Defaults to stdpath('data')/notex/notex.db
auto_index = true, -- Auto-index markdown files on save
index_on_startup = false, -- Index entire workspace on startup
-- File handling
max_file_size = 10 * 1024 * 1024, -- 10MB max file size
-- Performance settings
performance = {
max_query_time = 5000, -- 5 seconds max query time
cache_size = 100, -- Number of cached queries
enable_caching = true -- Enable query caching
},
-- UI settings
ui = {
border = "rounded", -- Window border style
max_width = 120, -- Max window width
max_height = 30, -- Max window height
show_help = true -- Show help in query results
},
-- Default view type
default_view_type = "table" -- "table", "list", or "grid"
})
```
## Query Syntax
Notex uses a simple, SQL-like query syntax designed for document discovery:
### Basic Structure
```
FROM documents
WHERE <conditions>
ORDER BY <field> <direction>
LIMIT <number>
```
### WHERE Conditions
```sql
-- Exact match
WHERE status = "published"
-- Partial match
WHERE tags LIKE "project"
-- Date comparison
WHERE created_at > "2023-01-01"
WHERE updated_at >= "2023-12-25"
-- Multiple conditions
WHERE status = "published" AND priority > 3
WHERE tags LIKE "urgent" OR status = "review"
```
### Ordering
```sql
-- Ascending (default)
ORDER BY title
-- Descending
ORDER BY created_at DESC
-- Multiple fields
ORDER BY priority DESC, created_at ASC
```
### Limiting Results
```sql
-- Limit number of results
LIMIT 10
-- Get most recent 5 documents
ORDER BY updated_at DESC LIMIT 5
```
## Document Metadata
Notex automatically extracts metadata from YAML frontmatter in your markdown files:
```markdown
---
title: "My Document"
status: "published"
priority: 5
tags: ["project", "urgent"]
due_date: "2023-12-25"
author: "John Doe"
---
# Document Content
Your markdown content goes here...
```
All YAML fields become queryable properties.
## Keybindings
Default keybindings (can be customized):
- `<leader>nq` - Show new query prompt
- `<leader>nr` - Show recent queries
- `<leader>ns` - Show saved queries
- `<leader>ni` - Index current workspace
- `<leader>nv` - Switch view type
- `<leader>ne` - Export current view
- `<leader>nc` - Cleanup database
In query result buffers:
- `<Enter>` - Open document
- `e` - Toggle inline editing
- `v` - Change view type
- `s` - Save query
- `r` - Refresh results
- `q` - Close buffer
## Query Results
Query results are displayed in virtual buffers with:
- **Interactive tables**: Sort, filter, and navigate results
- **Document preview**: Quick document information
- **Inline editing**: Edit document properties directly
- **Export options**: Save results as CSV, JSON, or Markdown
### View Types
1. **Table View**: Structured data display
2. **List View**: Compact document list
3. **Grid View**: Card-based layout
## API Reference
### Core Functions
```lua
-- Execute a query
local result = require('notex').execute_query_and_show_results(query_string)
-- Index a directory
require('notex').index_directory(path, options)
-- Get document details
local details = require('notex.index').get_document_details(document_id)
-- Save a query
require('notex.query').save_query(name, query_string)
-- Get statistics
local stats = require('notex').get_statistics()
```
### Query Engine
```lua
local query = require('notex.query')
-- Execute query
local result = query.execute_query(query_string)
-- Validate query syntax
local validation = query.validate_query_syntax(query_string)
-- Get suggestions
local suggestions = query.get_suggestions(partial_query, cursor_pos)
```
### Index Management
```lua
local index = require('notex.index')
-- Index documents
local result = index.index_documents(directory_path, options)
-- Search documents
local results = index.search_documents(criteria)
-- Get index statistics
local stats = index.get_statistics()
-- Validate index
local validation = index.validate_index()
```
## Performance
Notex is optimized for performance:
- **Multi-tier caching**: Memory, LRU, and timed caches
- **Incremental indexing**: Only process changed files
- **Database optimization**: SQLite with proper indexing
- **Lazy loading**: Load document details on demand
### Cache Configuration
```lua
require('notex').setup({
performance = {
enable_caching = true,
cache_size = 200, -- Increase for better performance
max_query_time = 10000 -- Allow longer queries for complex datasets
}
})
```
## Troubleshooting
### Common Issues
1. **Database locked errors**:
- Ensure you have proper file permissions
- Check if another Neovim instance is using the database
2. **Slow queries**:
- Increase `cache_size` in configuration
- Use more specific WHERE conditions
- Check database file size and consider cleanup
3. **Missing documents in results**:
- Run `:lua require('notex').index_workspace()` to reindex
- Check file permissions and YAML syntax
### Debug Mode
Enable debug logging:
```lua
vim.g.notex_debug = true
```
Check logs at `:lua print(vim.fn.stdpath('data') .. '/notex/notex.log')`
## Development
### Running Tests
```bash
# Install dependencies
luarocks install busted
luarocks install luafilesystem
# Run tests
busted tests/
```
### Project Structure
```
notex.nvim/
├── lua/notex/
│ ├── database/ # Database layer
│ ├── parser/ # Document parsing
│ ├── query/ # Query engine
│ ├── ui/ # User interface
│ ├── index/ # Document indexing
│ └── utils/ # Utilities
├── tests/
│ ├── unit/ # Unit tests
│ ├── integration/ # Integration tests
│ └── performance/ # Performance tests
└── specs/ # Specifications
```
### Contributing
1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Update documentation
6. Submit a pull request
## License
MIT License - see [LICENSE](LICENSE) file for details.
## Acknowledgments
- Inspired by Notion's database functionality
- Built with [lsqlite3](https://github.com/keplerproject/lua-sqlite3) for database operations
- UI components based on Neovim's native buffer API
- Testing with [busted](https://lunarmodules.github.io/busted/)
## Support
- 🐛 Report bugs: [GitHub Issues](https://github.com/your-username/notex.nvim/issues)
- 💡 Feature requests: [GitHub Discussions](https://github.com/your-username/notex.nvim/discussions)
- 📖 Documentation: [Wiki](https://github.com/your-username/notex.nvim/wiki)
---
Made with ❤️ for the Neovim community