Skip to content

feat/formula generated column #1694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 271 commits into
base: develop
Choose a base branch
from
Draft

feat/formula generated column #1694

wants to merge 271 commits into from

Conversation

nichenqin
Copy link
Contributor

@nichenqin nichenqin commented Jul 29, 2025 •
edited
Loading

Formula

  • Common

    • Convert teable formula to pg/sqlite formula
    • Convert unit test
    • Convert e2e test

  • Generated Column Handling on Field Creation

    • Automatically create corresponding database generated column when creating a formula field
    • Handle field dependencies to ensure generated columns are created only when dependencies exist
    • Set correct column metadata and display configuration for newly created generated columns
    • Handle duplicate formula generated columns
    • Update duplicated formula column should update legacy formula record values
    • handle scenarios "generation expression is not immutable"
    • Support create a lookup formula field

  • Handling on Field Deletion

    • Handle formula columns when dependent fields are deleted
    • Handle deletion of any field in multi-level dependencies

  • Generated Column Sync on Field Update

    • Regenerate database column when formula expression is updated
    • Handle deletion and recreation of generated columns when field type changes
    • Cascade update related generated columns when dependency fields change

  • Database Query Optimization

    • Prefer using generated column data during queries instead of real-time computation
    • Ensure data consistency between generated columns and original columns

  • Column Metadata Management

    • Display name and formatting rules for generated columns
    • Filtering support for generated columns in views
    • Sorting support for generated columns in views
    • Group support for generated columns in views

  • Handle event streaming

    • Publish event op with generated column field when update record
    • Publish event op with generated column field when create field
    • Publish event op with generated column field when duplicate field
    • Publish event op with generated column field when delete dependency field

  • Performance Optimization

    • Handle generated columns during batch operations
    • Strategy for creating generated columns in large data tables

  • Field Reference

    • Support reference another link field
    • Support reference another lookup field
    • Support reference another rollup field

Link Field

  • Support select field with junction table or fk
  • Support update link field value
  • Support convert one way to two way link
  • Fix two way to one way link issue
  • Support duplicate link field
  • Support link lookup a formula field
  • Fix link field order
    • Many Many
    • Many One
    • One Many
    • One One
  • Handle link -> formula -> link
  • Support field nested filter

Lookup Field

  • Support select lookup field any group to json
  • Support lookup generated formula field
  • Support lookup non generated formula field
  • Support lookup link field
  • Support recursively lookup table
  • Support field nested filter

Rollup Field

  • Support rollup field
  • Support field nested filter

Query

  • Support filter on virtual fields
  • Test all field filter
  • Support aggregate filter like count
  • Support sort on virtual fields
  • Test all field sort
  • Support group virtual fields
  • Test all field group
  • Support group field aggregate
  • Support field search
  • Fix group on rollup field

Bugs

  • formula -> rollup -> rollup generate sql error
  • formula -> recordId/autoNumber
  • table1/formula(->lookup) -> table1/linktable2(->table2/linktable1) -> table1/lookup -> table2 query build error

@nichenqin nichenqin marked this pull request as draft July 29, 2025 07:58
@nichenqin nichenqin added the preview This issue is already ready for preview label Jul 29, 2025
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

4 similar comments
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@nichenqin nichenqin force-pushed the feat/generated-column branch from 5e365fe to 31521dc Compare July 30, 2025 06:15
@nichenqin nichenqin changed the title feat/generated column feat/formula generated column Jul 30, 2025
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

1 similar comment
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

github-advanced-security[bot]
github-advanced-security bot found potential problems Jul 31, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

2 similar comments
@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

Deployment Status: ❌ Failed

@nichenqin nichenqin force-pushed the feat/generated-column branch from d46867e to efcc56c Compare August 1, 2025 06:45
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

5 similar comments
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 1, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@nichenqin nichenqin force-pushed the feat/generated-column branch from 27b5414 to b08b767 Compare August 2, 2025 05:37
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2025

Deployment Status: ❌ Failed

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

4 similar comments
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 3, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 4, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 4, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

@nichenqin
feat: realtime delete field op
592c33f
@nichenqin
fix: fix delete field and all dependent fields should be marked as error
12dde08
@nichenqin
feat: add formula field core hasUnresolvedReferences
1515c1f
@nichenqin
feat: create record modify module
3b0c881
@nichenqin
feat: modify module add delete record
574944b
@nichenqin
fix: fix delete record issue
c6d5e10
@nichenqin
fix: fix update record
b77a273
@nichenqin
chore: removed record calculation service
d41c600
@nichenqin
test: add create duplicate record test
6d2ea6c
@nichenqin
fix: fix field test
7ff7b53
@nichenqin
chore: fix duplicate field
99b3857
@nichenqin
refactor: modify service
f0588c4
@nichenqin
fix: fix lint issue
47f1ab9
@nichenqin
fix: missing sqlite prisma generate
5fa7a0f
@nichenqin
chore: generate db view name migrate for sqlite
ffd95a8
@nichenqin
fix: postgres prisma schema
9d461dc
@nichenqin
refactor: remove calculation code
a1c3688
@nichenqin
chore: sqlite bugs
7aa3573
@nichenqin
feat: implement PostgreSQL and SQLite record query dialects
6ecf54a
@nichenqin
feat: record query builder respect projection
2fec1b2
@nichenqin
feat: createRecordAggregateBuilder support projection
7978096
@nichenqin
fix: fix sqlite junction table issue
dc6f198
@nichenqin
feat: add computed service to calc ops
dfd9d7b
@nichenqin
fix: fix nested lookup for formula
8863a31
@nichenqin
fix: fix impact collector
74263f0
@nichenqin
chore: disable sqlite test
a8f5c97
@nichenqin
test: add computed orchestractor test
ce985a6
@nichenqin nichenqin force-pushed the feat/generated-column branch from 285b433 to ce985a6 Compare September 6, 2025 05:01
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

1 similar comment
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 6, 2025

Deployment Status: ✅ Success
🔗 Preview URL: https://pr-1694.sealoshzh.site

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
preview This issue is already ready for preview
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nichenqin @tea-artist