Content
Updated by Markus Kahl 18 days ago
Source: https://github.com/opf/openproject/pull/14525#issuecomment-2018240127
### Steps to reproduce
1. Start OpenProject 13
1. `docker run --rm -p 8080:80 -e OPENPROJECT_HTTPS=false -v /tmp/oppgdata:/var/openproject/pgdata -it openproject/community:13` Run the migration `20231123111357 CreateCustomFieldSections` for the first time.
2. Create a project custom field in the demo project under [http://localhost:8080/](http://localhost:8080/)
3. Stop OpenProject 13 and upgrade to and run 14 (current dev)
1. `docker run --rm -p 8080:80 -e OPENPROJECT_HTTPS=false -v /tmp/oppgdata:/var/openproject/pgdata --pull always -it openproject/community:dev`
4. Open the demo project overview page The columns `custom_field_section_id` is not filled
### What is the buggy behavior?
* You get an internal server error opening the page
* (The The columns `custom_field_section_id` is not filled for the existing `ProjectCustomFields`.) `ProjectCustomFields`.
### What is the expected behavior?
1. You should not get an error and see the custom field under the project custom fields section
1. (The The columns `custom_field_section_id` should be filled for the existing `ProjectCustomFields` after the migration finished.) finished.
### Additional info
```shell
Migrating to CreateCustomFieldSections (20231123111357)
== 20231123111357 CreateCustomFieldSections: migrating ========================
-- create_table(:custom_field_sections)
TRANSACTION (0.1ms) BEGIN
(4.7ms) CREATE TABLE "custom_field_sections" ("id" bigserial primary key, "position" integer, "name" character varying, "type" character varying, "created_at" timestamptz(6) NOT NULL, "updated_at" timestamptz(6) NOT NULL)
-> 0.0051s
-- add_reference(:custom_fields, :custom_field_section)
(0.3ms) ALTER TABLE "custom_fields" ADD "custom_field_section_id" bigint
(1.3ms) CREATE INDEX "index_custom_fields_on_custom_field_section_id" ON "custom_fields" ("custom_field_section_id")
-> 0.0046s
-- add_column(:custom_fields, :position_in_custom_field_section, :integer, {:null=>true})
(0.2ms) ALTER TABLE "custom_fields" ADD "position_in_custom_field_section" integer
-> 0.0006s
CustomFieldSection Load (0.5ms) SELECT "custom_field_sections".* FROM "custom_field_sections" WHERE "custom_field_sections"."type" = $1 AND ("custom_field_sections"."position" IS NOT NULL) ORDER BY "custom_field_sections"."position" DESC LIMIT $2 [["type", "ProjectCustomFieldSection"], ["LIMIT", 1]]
ProjectCustomFieldSection Create (0.6ms) INSERT INTO "custom_field_sections" ("position", "name", "type", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["position", 1], ["name", "Project attributes"], ["type", "ProjectCustomFieldSection"], ["created_at", "2024-03-25 14:53:54.495725"], ["updated_at", "2024-03-25 14:53:54.495725"]]
ProjectCustomField Load (0.7ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1000]]
CustomOption Load (0.6ms) SELECT "custom_options".* FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13]]
CustomField Pluck (0.2ms) SELECT "custom_fields"."name" FROM "custom_fields" WHERE "custom_fields"."type" = $1 AND "custom_fields"."id" != $2 [["type", "ProjectCustomField"], ["id", 13]]
↳ app/models/custom_field.rb:57:in `uniqueness_of_name_with_scope'
CustomOption Pluck (0.2ms) SELECT "custom_options"."id" FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 AND "custom_options"."default_value" = $2 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13], ["default_value", true]]
↳ app/models/custom_field.rb:83:in `default_value'
ProjectCustomField Load (1.3ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 1], ["LIMIT", 1]]
ProjectCustomField Update (0.8ms) UPDATE "custom_fields" SET "updated_at" = $1 WHERE "custom_fields"."id" = $2 [["updated_at", "2024-03-25 14:53:54.548179"], ["id", 13]]
ProjectCustomField Load (0.4ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 1], ["LIMIT", 1]]
ProjectCustomField Count (0.3ms) SELECT COUNT(*) FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" = 1) [["custom_field_section_id", 1]]
== 20231123111357 CreateCustomFieldSections: migrated (0.0813s) ===============
ActiveRecord::SchemaMigration Create (0.2ms) INSERT INTO "schema_migrations" ("version") VALUES ('20231123111357') RETURNING "version"
TRANSACTION (0.7ms) COMMIT
```
> Nice work, but I found when first doing the CreateCustomFieldSections migrate, existing `ProjectCustomField` (in this case, custom\_fields.id==13) the columns custom\_field\_section\_id is not filled, maybe due to custom\_field\_section is just added.
>
> So I re-run below code and got the custom\_field\_section\_id updated.
>
>
```ruby
[2] pry(main)> ProjectCustomField.first
ProjectCustomField Load (0.6ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1]]
=> #<ProjectCustomField:0x0000000134610960
id: 13,
type: "ProjectCustomField",
field_format: "list",
regexp: "",
min_length: 0,
max_length: 0,
is_required: false,
is_for_all: false,
is_filter: false,
position: 1,
searchable: true,
editable: true,
visible: true,
multi_value: false,
default_value: nil,
name: "测试项目字段",
created_at: Thu, 25 Jan 2024 13:53:11.813609000 CST +08:00,
updated_at: Mon, 25 Mar 2024 22:53:54.548179000 CST +08:00,
content_right_to_left: false,
allow_non_open_versions: false,
custom_field_section_id: nil,
position_in_custom_field_section: nil>
[3] pry(main)> section = ProjectCustomFieldSection.create!(
[3] pry(main)* name: "Project attributes 2"
[3] pry(main)* )
TRANSACTION (0.1ms) BEGIN
CustomFieldSection Load (1.2ms) SELECT "custom_field_sections".* FROM "custom_field_sections" WHERE "custom_field_sections"."type" = $1 AND ("custom_field_sections"."position" IS NOT NULL) ORDER BY "custom_field_sections"."position" DESC LIMIT $2 [["type", "ProjectCustomFieldSection"], ["LIMIT", 1]]
ProjectCustomFieldSection Create (0.5ms) INSERT INTO "custom_field_sections" ("position", "name", "type", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["position", 2], ["name", "Project attributes 2"], ["type", "ProjectCustomFieldSection"], ["created_at", "2024-03-25 14:56:14.169673"], ["updated_at", "2024-03-25 14:56:14.169673"]]
TRANSACTION (0.9ms) COMMIT
=> #<ProjectCustomFieldSection:0x0000000169bb6b78
id: 2,
position: 2,
name: "Project attributes 2",
type: "ProjectCustomFieldSection",
created_at: Mon, 25 Mar 2024 22:56:14.169673000 CST +08:00,
updated_at: Mon, 25 Mar 2024 22:56:14.169673000 CST +08:00>
[4] pry(main)> ProjectCustomField.find_each do |project_custom_field|
[4] pry(main)* project_custom_field.update!(custom_field_section_id: section.id)
[4] pry(main)* end
ProjectCustomField Load (0.4ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1000]]
TRANSACTION (3.6ms) BEGIN
CustomOption Load (0.7ms) SELECT "custom_options".* FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13]]
CustomField Pluck (0.3ms) SELECT "custom_fields"."name" FROM "custom_fields" WHERE "custom_fields"."type" = $1 AND "custom_fields"."id" != $2 [["type", "ProjectCustomField"], ["id", 13]]
CustomOption Pluck (0.2ms) SELECT "custom_options"."id" FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 AND "custom_options"."default_value" = $2 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13], ["default_value", true]]
ProjectCustomField Load (0.3ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "custom_fields"."position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 2], ["LIMIT", 1]]
ProjectCustomField Update (0.4ms) UPDATE "custom_fields" SET "updated_at" = $1, "custom_field_section_id" = $2, "position_in_custom_field_section" = $3 WHERE "custom_fields"."id" = $4 [["updated_at", "2024-03-25 14:56:29.008077"], ["custom_field_section_id", 2], ["position_in_custom_field_section", 1], ["id", 13]]
ProjectCustomField Load (0.2ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "custom_fields"."position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 2], ["LIMIT", 1]]
ProjectCustomField Count (0.2ms) SELECT COUNT(*) FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" = 1) [["custom_field_section_id", 2]]
TRANSACTION (1.3ms) COMMIT
=> nil
```
> I try at TH staging server, it will lead 500 error due to custom\_field\_section\_id is NULL. I try running above code to fix and it works.
### Steps to reproduce
1. Start OpenProject 13
1. `docker run --rm -p 8080:80 -e OPENPROJECT_HTTPS=false -v /tmp/oppgdata:/var/openproject/pgdata -it openproject/community:13`
2. Create a project custom field in the demo project under [http://localhost:8080/](http://localhost:8080/)
3. Stop OpenProject 13 and upgrade to and run 14 (current dev)
1. `docker run --rm -p 8080:80 -e OPENPROJECT_HTTPS=false -v /tmp/oppgdata:/var/openproject/pgdata --pull always -it openproject/community:dev`
4. Open the demo project overview page
### What is the buggy behavior?
* You get an internal server error opening the page
* (The
### What is the expected behavior?
1. You should not get an error and see the custom field under the project custom fields section
1. (The
### Additional info
```shell
Migrating to CreateCustomFieldSections (20231123111357)
== 20231123111357 CreateCustomFieldSections: migrating ========================
-- create_table(:custom_field_sections)
TRANSACTION (0.1ms) BEGIN
(4.7ms) CREATE TABLE "custom_field_sections" ("id" bigserial primary key, "position" integer, "name" character varying, "type" character varying, "created_at" timestamptz(6) NOT NULL, "updated_at" timestamptz(6) NOT NULL)
-> 0.0051s
-- add_reference(:custom_fields, :custom_field_section)
(0.3ms) ALTER TABLE "custom_fields" ADD "custom_field_section_id" bigint
(1.3ms) CREATE INDEX "index_custom_fields_on_custom_field_section_id" ON "custom_fields" ("custom_field_section_id")
-> 0.0046s
-- add_column(:custom_fields, :position_in_custom_field_section, :integer, {:null=>true})
(0.2ms) ALTER TABLE "custom_fields" ADD "position_in_custom_field_section" integer
-> 0.0006s
CustomFieldSection Load (0.5ms) SELECT "custom_field_sections".* FROM "custom_field_sections" WHERE "custom_field_sections"."type" = $1 AND ("custom_field_sections"."position" IS NOT NULL) ORDER BY "custom_field_sections"."position" DESC LIMIT $2 [["type", "ProjectCustomFieldSection"], ["LIMIT", 1]]
ProjectCustomFieldSection Create (0.6ms) INSERT INTO "custom_field_sections" ("position", "name", "type", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["position", 1], ["name", "Project attributes"], ["type", "ProjectCustomFieldSection"], ["created_at", "2024-03-25 14:53:54.495725"], ["updated_at", "2024-03-25 14:53:54.495725"]]
ProjectCustomField Load (0.7ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1000]]
CustomOption Load (0.6ms) SELECT "custom_options".* FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13]]
CustomField Pluck (0.2ms) SELECT "custom_fields"."name" FROM "custom_fields" WHERE "custom_fields"."type" = $1 AND "custom_fields"."id" != $2 [["type", "ProjectCustomField"], ["id", 13]]
↳ app/models/custom_field.rb:57:in `uniqueness_of_name_with_scope'
CustomOption Pluck (0.2ms) SELECT "custom_options"."id" FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 AND "custom_options"."default_value" = $2 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13], ["default_value", true]]
↳ app/models/custom_field.rb:83:in `default_value'
ProjectCustomField Load (1.3ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 1], ["LIMIT", 1]]
ProjectCustomField Update (0.8ms) UPDATE "custom_fields" SET "updated_at" = $1 WHERE "custom_fields"."id" = $2 [["updated_at", "2024-03-25 14:53:54.548179"], ["id", 13]]
ProjectCustomField Load (0.4ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 1], ["LIMIT", 1]]
ProjectCustomField Count (0.3ms) SELECT COUNT(*) FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" = 1) [["custom_field_section_id", 1]]
== 20231123111357 CreateCustomFieldSections: migrated (0.0813s) ===============
ActiveRecord::SchemaMigration Create (0.2ms) INSERT INTO "schema_migrations" ("version") VALUES ('20231123111357') RETURNING "version"
TRANSACTION (0.7ms) COMMIT
> Nice work, but I found when first doing the CreateCustomFieldSections migrate, existing `ProjectCustomField` (in this case, custom\_fields.id==13) the columns custom\_field\_section\_id is not filled, maybe due to custom\_field\_section is just added.
>
> So I re-run below code and got the custom\_field\_section\_id updated.
>
>
[2] pry(main)> ProjectCustomField.first
ProjectCustomField Load (0.6ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1]]
=> #<ProjectCustomField:0x0000000134610960
id: 13,
type: "ProjectCustomField",
field_format: "list",
regexp: "",
min_length: 0,
max_length: 0,
is_required: false,
is_for_all: false,
is_filter: false,
position: 1,
searchable: true,
editable: true,
visible: true,
multi_value: false,
default_value: nil,
name: "测试项目字段",
created_at: Thu, 25 Jan 2024 13:53:11.813609000 CST +08:00,
updated_at: Mon, 25 Mar 2024 22:53:54.548179000 CST +08:00,
content_right_to_left: false,
allow_non_open_versions: false,
custom_field_section_id: nil,
position_in_custom_field_section: nil>
[3] pry(main)> section = ProjectCustomFieldSection.create!(
[3] pry(main)* name: "Project attributes 2"
[3] pry(main)* )
TRANSACTION (0.1ms) BEGIN
CustomFieldSection Load (1.2ms) SELECT "custom_field_sections".* FROM "custom_field_sections" WHERE "custom_field_sections"."type" = $1 AND ("custom_field_sections"."position" IS NOT NULL) ORDER BY "custom_field_sections"."position" DESC LIMIT $2 [["type", "ProjectCustomFieldSection"], ["LIMIT", 1]]
ProjectCustomFieldSection Create (0.5ms) INSERT INTO "custom_field_sections" ("position", "name", "type", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["position", 2], ["name", "Project attributes 2"], ["type", "ProjectCustomFieldSection"], ["created_at", "2024-03-25 14:56:14.169673"], ["updated_at", "2024-03-25 14:56:14.169673"]]
TRANSACTION (0.9ms) COMMIT
=> #<ProjectCustomFieldSection:0x0000000169bb6b78
id: 2,
position: 2,
name: "Project attributes 2",
type: "ProjectCustomFieldSection",
created_at: Mon, 25 Mar 2024 22:56:14.169673000 CST +08:00,
updated_at: Mon, 25 Mar 2024 22:56:14.169673000 CST +08:00>
[4] pry(main)> ProjectCustomField.find_each do |project_custom_field|
[4] pry(main)* project_custom_field.update!(custom_field_section_id: section.id)
[4] pry(main)* end
ProjectCustomField Load (0.4ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."type" = $1 ORDER BY "custom_fields"."id" ASC LIMIT $2 [["type", "ProjectCustomField"], ["LIMIT", 1000]]
TRANSACTION (3.6ms) BEGIN
CustomOption Load (0.7ms) SELECT "custom_options".* FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13]]
CustomField Pluck (0.3ms) SELECT "custom_fields"."name" FROM "custom_fields" WHERE "custom_fields"."type" = $1 AND "custom_fields"."id" != $2 [["type", "ProjectCustomField"], ["id", 13]]
CustomOption Pluck (0.2ms) SELECT "custom_options"."id" FROM "custom_options" WHERE "custom_options"."custom_field_id" = $1 AND "custom_options"."default_value" = $2 ORDER BY "custom_options"."position" ASC [["custom_field_id", 13], ["default_value", true]]
ProjectCustomField Load (0.3ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "custom_fields"."position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 2], ["LIMIT", 1]]
ProjectCustomField Update (0.4ms) UPDATE "custom_fields" SET "updated_at" = $1, "custom_field_section_id" = $2, "position_in_custom_field_section" = $3 WHERE "custom_fields"."id" = $4 [["updated_at", "2024-03-25 14:56:29.008077"], ["custom_field_section_id", 2], ["position_in_custom_field_section", 1], ["id", 13]]
ProjectCustomField Load (0.2ms) SELECT "custom_fields".* FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" IS NOT NULL) ORDER BY "custom_fields"."position_in_custom_field_section" DESC LIMIT $2 [["custom_field_section_id", 2], ["LIMIT", 1]]
ProjectCustomField Count (0.2ms) SELECT COUNT(*) FROM "custom_fields" WHERE "custom_fields"."custom_field_section_id" = $1 AND ("custom_fields"."position_in_custom_field_section" = 1) [["custom_field_section_id", 2]]
TRANSACTION (1.3ms) COMMIT
=> nil
```
> I try at TH staging server, it will lead 500 error due to custom\_field\_section\_id is NULL. I try running above code to fix and it works.