Error dirty database version 1 fix and force version

the error appears as below when migrate timeout: Dirty database version 1. Fix and force version Steps to Reproduce Steps to reproduce the behavior: do migrate timeout migrate again Expected Behavi...

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account


Closed

duyanghao opened this issue

Sep 4, 2019

· 23 comments

Comments

@duyanghao

the error appears as below when migrate timeout:

Dirty database version 1. Fix and force version

Steps to Reproduce
Steps to reproduce the behavior:

  1. do migrate timeout
  2. migrate again

Expected Behavior
migrate again could automatically solve this problem and do migrate normally.

This was referenced

Sep 4, 2019

@dhui

Timeouts may occur and the correct fix is not obvious, so user intervention is necessary.
e.g. The migration could have never applied, partially applied, or fully applied

@duyanghao

@dhui How to fix Dirty database status? Is there any documents for this?

@christopher-kleine

@duyanghao Force to a previous version and then run «up» again.

So if you get an error in version 10, fix the error and then force it to version 9. Afterwards you can migrate again.

@duyanghao

@duyanghao Force to a previous version and then run «up» again.

So if you get an error in version 10, fix the error and then force it to version 9. Afterwards you can migrate again.

@anihex Could you please give an example of fixing Dirty database status as described above?

@christopher-kleine

@duyanghao Of course I can.

Let’s assume I want to create a new table. But I made a «slight» mistake (missing Primary Key).

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Now I try to migrate it:

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable up

migrate will refuse this:

error: 2 errors occurred:
	* migration failed: there is no primary key for referenced table "auth_users" in line 0: BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;
 (details: pq: there is no primary key for referenced table "auth_users")
	* pq: current transaction is aborted, commands ignored until end of transaction block in line 0: SELECT pg_advisory_unlock($1)

Now I fixed it and added the PRIMARY KEY to the migration:

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL PRIMARY KEY,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

If I try to migrate again, migrate will STILL refuse:

error: Dirty database version 16. Fix and force version.

So I have to go the last successfull version, which is 15.

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable force 15

If i migrate again, the output will now be:

16/u create_auth_users (17.608619ms)

The same works if you want to go down. But if would be 17 in this case, not 15.
I hope this helps a bit.

diegotf30, dmerrick, nikoren, JBarna, kurtpeek, klipspringerc, vmpartner, codingpop, pallavJha, denverSerraoWP, and 67 more reacted with thumbs up emoji
duyng404, AlexeyInc, EdwinWalela, alfredfrancis, 123vivekr, 5t4lk, and ekowcharles reacted with rocket emoji

@duyanghao

@duyanghao Of course I can.

Let’s assume I want to create a new table. But I made a «slight» mistake (missing Primary Key).

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Now I try to migrate it:

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable up

migrate will refuse this:

error: 2 errors occurred:
	* migration failed: there is no primary key for referenced table "auth_users" in line 0: BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;
 (details: pq: there is no primary key for referenced table "auth_users")
	* pq: current transaction is aborted, commands ignored until end of transaction block in line 0: SELECT pg_advisory_unlock($1)

Now I fixed it and added the PRIMARY KEY to the migration:

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL PRIMARY KEY,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

If I try to migrate again, migrate will STILL refuse:

error: Dirty database version 16. Fix and force version.

So I have to go the last successfull version, which is 15.

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable force 15

If i migrate again, the output will now be:

16/u create_auth_users (17.608619ms)

The same works if you want to go down. But if would be 17 in this case, not 15.
I hope this helps a bit.

@anihex Thank for your reply, that really helps a lot.

selmison, jsagl, jsgv, JoshEvan, boz14676, farwish, Sushanthbc, jtikon, BlueTigress, pwli0755, and 3 more reacted with thumbs up emoji

@bkakadiya42

Maybe I am missing something here. If we are running migrations within a transaction, why do we need to mark the database as dirty? Asking this because it slows down rapid iterations on the migration if the developer has to force after every failure, and when it is the very first migration you can’t force it to the version 000000

tonyhb, ann-kilzer, warent, Streppel, huangc28, tiagosimao, ryanrolds, ninjahoahong, tkshnwesper, nickzylstra, and 22 more reacted with thumbs up emoji
jtikon, macnibblet, and LouisSayers reacted with confused emoji

@ann-kilzer

@bkakadiya42 I had the same issue, ended up resetting my entire DB to start clean and then fix my migration file. Hope that helps!

@warent

It helps bandaid the problem, but it still leaves the question open why a failed transaction makes the database dirty. This must be a bug. The «force» UX is also very user hostile.

bkakadiya42, spangenberg, MaikelH, rustagram, shinto-dev, CDRayn, donkeysharp, mnvx, xrn, dankobgd, and 9 more reacted with thumbs up emoji

@tiagosimao

Maybe I am missing something here. If we are running migrations within a transaction, why do we need to mark the database as dirty? Asking this because it slows down rapid iterations on the migration if the developer has to force after every failure, and when it is the very first migration you can’t force it to the version 000000

Spot on. Error on first migration seems irrecoverable through migrate‘s interface. I will hack away and drop the schema_migrations table (if dirty && Version==1) but this is clearly an abstraction violation.
Also, all of this is just silly when using Postgres (or any other db with transaction support in schema changes for that matter)

Didn’t understood why the issue is marked as resolved, even the initial question refers to error on version 1 but all answers just seem to ignore that detail. Maybe I’m missing something as well. The docs don’t seem to scratch this itch tough.

Some possible ways this lib could help the developer in tackling this corner case I can think of:

  • Allowing disabling the dirty feature
  • Adding method to drop schema_migrations table
  • Never setting first version as dirty (probably incoherent, bad idea)

All for helping out, just didn’t fully understood if this is something recognized as a problem.
I really (really) don’t want to sound like a jerk, but this bug makes the lib unusable in production. I cannot have an application that out of the blue requires a human messing with the DB.

Cheerio!

@nnachefski

I’m hitting this while deploying harbor with the kube operator.

2020-09-03T14:39:47Z [INFO] [/common/dao/pgsql.go:127]: Upgrading schema for pgsql ...
2020-09-03T14:39:47Z [ERROR] [/common/dao/pgsql.go:132]: Failed to upgrade schema, error: "Dirty database version 10. Fix and force version."
2020-09-03T14:39:47Z [FATAL] [/core/main.go:123]: failed to migrate: Dirty database version 10. Fix and force version.

@dhui

If we are running migrations within a transaction, why do we need to mark the database as dirty?

still leaves the question open why a failed transaction makes the database dirty

migrate does not enforce this (e.g. does not run all migrations in a transaction) so it cannot automatically rollback upon failure. If all of your migrations are wrapped in a transaction, then you can safely force the migration to the previous applied migration. Otherwise, you’ll need to inspect the state of your db and determine how to proceed.

The «force» UX is also very user hostile.

Even if migrate were able to rollback failed migration, user intervention is likely required to fix some state for a successful migration. e.g. re-running the same migration with the same starting state won’t succeed
Letting the developer/sysadmin know that something is broken and requires their attention and having them acknowledge that the issue has been resolved is a good thing. However, I agree that the UX could be smoother. e.g. forcing is an extra step vs just trying to re-apply the migration

@dhui

@mnvx

I agree with @tiagosimao, would be great to have option

Allowing disabling the dirty feature

It is not handy to work with dirty migrations on production. In my case migrations executes from docker container with code.

  • This container cant start until migrations are successfully executed.
  • So it is not possible to run force command in this container.
  • I see here two ways and they are both bad (because it is a bit of hell for admins and more large time of services unavailability):
    • adapt ci/cd processes to support force command,
    • change version directly in database

And my database is Posgtes where DDL queries are support transactions… So best way in my case is as said above

Allowing disabling the dirty feature

@xrn

Regarding that topic also I would like to speak following:

If my migration failed during the Down execution — let’s say I would like to rollback step 5 and back to migration 4 and it crash. I will see Dirty database version 5. Fix and force version — Without information regarding the direction even if I am using transactions for each migration I do not know if I should force it to version 4 (this will be a case for broken Up) or force it to version 5 (broken Down)

Currently, there is a blocker for new migrations if there is a dirty flag — thanks to that we can not mess up with db state (making several up and down). What do you think about adding information about the direction which caused that dirty state? Thanks to that user can automate forcing to correct version if is sure that is using transactions. It will be very helpful IMO

@joncodo

If the admins can make a decision here, I will implement it. This is really not a good design and is blocking new users from adoption.

2021/12/15 22:28:06 Applying all down migrations
2021/12/15 22:28:06 error: Dirty database version 1. Fix and force version.

@kskenyon

I’m trying to install Chirpstack with Postgresql on Debian Buster. I’ll be danged if I can get past this error when starting the application-server.

level=fatal msg=»setup storage error: storage: migrate up error: Dirty database version 27. Fix and force version.»

No idea what to do and where to do it. This must be a common problem. Golang related?

@joncodo

I’m trying to install Chirpstack with Postgresql on Debian Buster. I’ll be danged if I can get past this error when starting the application-server.

level=fatal msg=»setup storage error: storage: migrate up error: Dirty database version 27. Fix and force version.»

No idea what to do and where to do it. This must be a common problem. Golang related?

Try this readme I made for our project: https://github.com/merico-dev/lake/blob/migrations-v2/MIGRATIONS.md

@kskenyon

Thanks. I managed to get it working.

Sent from my iPad

On Jan 14, 2022, at 7:12 AM, joncodo ***@***.***> wrote:


I’m trying to install Chirpstack with Postgresql on Debian Buster. I’ll be danged if I can get past this error when starting the application-server.

level=fatal msg=»setup storage error: storage: migrate up error: Dirty database version 27. Fix and force version.»

No idea what to do and where to do it. This must be a common problem. Golang related?

Try this readme I made for our project: https://github.com/merico-dev/lake/blob/migrations-v2/MIGRATIONS.md


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.

@buraksecer

the error appears as below when migrate timeout:

Dirty database version 1. Fix and force version

Steps to Reproduce Steps to reproduce the behavior:

  1. do migrate timeout
  2. migrate again

Expected Behavior migrate again could automatically solve this problem and do migrate normally.

If you use docker for database, you have to clear database volumes, this is working for me.

@CodeNinjaUG

this is what worked for me
migrate -path db/migrations -database «postgresql://root:8bdc7axyzex@localhost:5432/simple_bank?sslmode=disable» force 1

i had to even drop a database for it work.. but the force version works well

@aksenof

i resolved this error by inserting version with zero dirty flag in the database

in my case, my database name was «schema»

so, check the dirty version:

SELECT * FROM schema_migrations;

in the output i found version = 1 and dirty = 1 and sequence = 1666689290970899911

then i insert new data:

INSERT INTO schema_migrations(version, dirty, sequence) VALUES (1, 0, 1666689299985899911);

congratulations, problem solved!

note: sequence — it’s a current datetime in nanoseconds, so, you need the new sequence is larger than the previous one

@dominikbraun

Sorry to be so pointed, but if this is really by design, the design is wrong.

Содержание

  1. Name already in use
  2. migrate / FAQ.md
  3. Users who have contributed to this file
  4. Name already in use
  5. migrate / GETTING_STARTED.md
  6. Users who have contributed to this file
  7. Discussion on: How to write & run database migration in Golang
  8. «Dirty database version» error. #217
  9. Comments
  10. Dirty database version 1. Fix and force version #282
  11. Comments

Name already in use

migrate / FAQ.md

  • Go to file T
  • Go to line L
  • Copy path
  • Copy permalink

5 contributors

Users who have contributed to this file

Copy raw contents

Copy raw contents

How is the code base structured?

Why is there no source/driver.go:Last() ?

It’s not needed. And unless the source has a «native» way to read a directory in reversed order, it might be expensive to do a full directory scan in order to get the last element.

What is a NilMigration? NilVersion?

NilMigration defines a migration without a body. NilVersion is defined as const -1.

What is the difference between uint(version) and int(targetVersion)?

version refers to an existing migration version coming from a source and therefore can never be negative. targetVersion can either be a version OR represent a NilVersion, which equals -1.

What’s the difference between Next/Previous and Up/Down?

Why two separate files (up and down) for a migration?

It makes all of our lives easier. No new markup/syntax to learn for users and existing database utility tools continue to work as expected.

How many migrations can migrate handle?

Whatever the maximum positive signed integer value is for your platform. For 32bit it would be 2,147,483,647 migrations. Migrate only keeps references to the currently run and pre-fetched migrations in memory. Please note that some source drivers need to do build a full «directory» tree first, which puts some heat on the memory consumption.

Are the table tests in migrate_test.go bloated?

Yes and no. There are duplicate test cases for sure but they don’t hurt here. In fact the tests are very visual now and might help new users understand expected behaviors quickly. Migrate from version x to y and y is the last migration? Just check out the test for that particular case and know what’s going on instantly.

What is Docker being used for?

Why not just use docker-compose?

It doesn’t give us enough runtime control for testing. We want to be able to bring up containers fast and whenever we want, not just once at the beginning of all tests.

Can I maintain my driver in my own repository?

Yes, technically thats possible. We want to encourage you to contribute your driver to this respository though. The driver’s functionality is dictated by migrate’s interfaces. That means there should really just be one driver for a database/ source. We want to prevent a future where several drivers doing the exact same thing, just implemented a bit differently, co-exist somewhere on GitHub. If users have to do research first to find the «best» available driver for a database in order to get started, we would have failed as an open source community.

Can I mix multiple sources during a batch of migrations?

What does «dirty» database mean?

Before a migration runs, each database sets a dirty flag. Execution stops if a migration fails and the dirty state persists, which prevents attempts to run more migrations on top of a failed migration. You need to manually fix the error and then «force» the expected version.

What happens if two programs try and update the database at the same time?

Database-specific locking features are used by some database drivers to prevent multiple instances of migrate from running migrations at the same time the same database at the same time. For example, the MySQL driver uses the GET_LOCK function, while the Postgres driver uses the pg_advisory_lock function.

Do I need to create a table for tracking migration version used?

No, it is done automatically.

Can I use migrate with a non-Go project?

Yes, you can use the migrate CLI in a non-Go project, but there are probably other libraries/frameworks available that offer better test and deploy integrations in that language/framework.

I have got an error Dirty database version 1. Fix and force version . What should I do?

Источник

Name already in use

migrate / GETTING_STARTED.md

  • Go to file T
  • Go to line L
  • Copy path
  • Copy permalink

3 contributors

Users who have contributed to this file

Copy raw contents

Copy raw contents

Before you start, you should understand the concept of forward/up and reverse/down database migrations.

Configure a database for your application. Make sure that your database driver is supported here.

Create some migrations using migrate CLI. Here is an example:

Once you create your files, you should fill them.

IMPORTANT: In a project developed by more than one person there is a chance of migrations inconsistency — e.g. two developers can create conflicting migrations, and the developer that created their migration later gets it merged to the repository first. Developers and Teams should keep an eye on such cases (especially during code review). Here is the issue summary if you would like to read more.

Consider making your migrations idempotent — we can run the same sql code twice in a row with the same result. This makes our migrations more robust. On the other hand, it causes slightly less control over database schema — e.g. let’s say you forgot to drop the table in down migration. You run down migration — the table is still there. When you run up migration again — CREATE TABLE would return an error, helping you find an issue in down migration, while CREATE TABLE IF NOT EXISTS would not. Use those conditions wisely.

In case you would like to run several commands/queries in one migration, you should wrap them in a transaction (if your database supports it). This way if one of commands fails, our database will remain unchanged.

Run your migrations through the CLI or your app and check if they applied expected changes. Just to give you an idea:

Just add the code to your app and you’re ready to go!

Before commiting your migrations you should run your migrations up, down, and then up again to see if migrations are working properly both ways. (e.g. if you created a table in a migration but reverse migration did not delete it, you will encounter an error when running the forward migration again) It’s also worth checking your migrations in a separate, containerized environment. You can find some tools at the end of this document.

IMPORTANT: If you would like to run multiple instances of your app on different machines be sure to use a database that supports locking when running migrations. Otherwise you may encounter issues.

Forcing your database version

In case you run a migration that contained an error, migrate will not let you run other migrations on the same database. You will see an error like Dirty database version 1. Fix and force version , even when you fix the erred migration. This means your database was marked as ‘dirty’. You need to investigate the migration error — was your migration applied partially, or was it not applied at all? Once you know, you should force your database to a version reflecting it’s real state. You can do so with force command:

Once you force the version and your migration was fixed, your database is ‘clean’ again and you can proceed with your migrations.

For details and example of usage see this comment.

Источник

Discussion on: How to write & run database migration in Golang

Getting this error while migratedown

«Dirty database version -1. Fix and force version.»

can anyone help with a solution! it will be a great help if you can.

Hi Apoorv, try this cmd first

migrate -path db/migration -database «postgresql://root:secret@localhost:5432/simple_bank?sslmode=disable» force 1

and then run makefile cmd

it should solve this issue. Here’s the docs for it
github.com/golang-migrate/migrate/.

For further actions, you may consider blocking this person and/or reporting abuse

DEV Community 👩‍💻👨‍💻 — A constructive and inclusive social network for software developers. With you every step of your journey.

Built on Forem — the open source software that powers DEV and other inclusive communities.

Made with love and Ruby on Rails. DEV Community 👩‍💻👨‍💻 © 2016 — 2023.

We’re a place where coders share, stay up-to-date and grow their careers.

Источник

«Dirty database version» error. #217

first of all, thanks for great project. I now trying to work with v3, and having some issues related to, well, understanding error messages. For example, during normal development workflow, creating new SQL-files, there was an error in one of them and migration failed. After fixing the error, on the next run, I got message:

Dirty database version 4. Fix and force version.

And the problem is it’s not clear what does mean «dirty database» in this context and how to «fix it» and what does mean «force version». I haven’t found explanation in the docs or issues. Of course, as it’s development env, I can just drop the db and recreate from scratch, but I suppose there should be «official» way to fix dirty database 🙂

PS. I’m running migrations from code, not from CLI, if it matters.

Thanks in advance!

UPD: After some digging, I ended up updating schema_migrations table manually, by UPDATE .. SET dirty=false and playing with version, but it’s not how it supposed to work, is it?

The text was updated successfully, but these errors were encountered:

or when using the cli:
migrate force V — Set version V but don’t run migration (ignores dirty state)

Does that answer your question? Can you help to make the docs more clear?

Thanks @mattes! It’s weird (or maybe not) how I didn’t find this part in FAQ, deliberately searching for the explanation of dirty state.

Generally I think what really should help is more verbose errors descriptions. Descriptions written with zero knowledge user reading them in mind. Like instead of «no change» it could be something like «Your database already is up to date and this command makes no change. Probably you wanted to run bla-bla-bla» . 🙂

In case of «Dirty database version. Fix and force version.» I guess it’s enough to tell how exactly to «force the version». Something like «After unsuccessful migration, database is marked as dirty at migration version V. Fix the issue and use ‘migrate force V’ to forcefully update version».

I’m not doing PR on this, because I’m sure I’m not aware of all use cases where these errors might occur and you can come up with much better error message text. But idea is that more verbose self-explanatory errors would be a huge step forward in usability of migrate .

Источник

Dirty database version 1. Fix and force version #282

the error appears as below when migrate timeout:

Steps to Reproduce
Steps to reproduce the behavior:

  1. do migrate timeout
  2. migrate again

Expected Behavior
migrate again could automatically solve this problem and do migrate normally.

The text was updated successfully, but these errors were encountered:

Timeouts may occur and the correct fix is not obvious, so user intervention is necessary.
e.g. The migration could have never applied, partially applied, or fully applied

@dhui How to fix Dirty database status ? Is there any documents for this?

@duyanghao Force to a previous version and then run «up» again.

So if you get an error in version 10, fix the error and then force it to version 9. Afterwards you can migrate again.

@duyanghao Force to a previous version and then run «up» again.

So if you get an error in version 10, fix the error and then force it to version 9. Afterwards you can migrate again.

@anihex Could you please give an example of fixing Dirty database status as described above?

@duyanghao Of course I can.

Let’s assume I want to create a new table. But I made a «slight» mistake (missing Primary Key).

Now I try to migrate it:

migrate will refuse this:

Now I fixed it and added the PRIMARY KEY to the migration:

If I try to migrate again, migrate will STILL refuse:

So I have to go the last successfull version, which is 15.

If i migrate again, the output will now be:

The same works if you want to go down. But if would be 17 in this case, not 15.
I hope this helps a bit.

Let’s assume I want to create a new table. But I made a «slight» mistake (missing Primary Key).

Now I try to migrate it:

migrate will refuse this:

Now I fixed it and added the PRIMARY KEY to the migration:

If I try to migrate again, migrate will STILL refuse:

So I have to go the last successfull version, which is 15.

If i migrate again, the output will now be:

The same works if you want to go down. But if would be 17 in this case, not 15.
I hope this helps a bit.

@anihex Thank for your reply, that really helps a lot.

Maybe I am missing something here. If we are running migrations within a transaction, why do we need to mark the database as dirty? Asking this because it slows down rapid iterations on the migration if the developer has to force after every failure, and when it is the very first migration you can’t force it to the version 000000

@bkakadiya42 I had the same issue, ended up resetting my entire DB to start clean and then fix my migration file. Hope that helps!

It helps bandaid the problem, but it still leaves the question open why a failed transaction makes the database dirty. This must be a bug. The «force» UX is also very user hostile.

Maybe I am missing something here. If we are running migrations within a transaction, why do we need to mark the database as dirty? Asking this because it slows down rapid iterations on the migration if the developer has to force after every failure, and when it is the very first migration you can’t force it to the version 000000

Spot on. Error on first migration seems irrecoverable through migrate‘s interface. I will hack away and drop the schema_migrations table (if dirty && Version==1) but this is clearly an abstraction violation.
Also, all of this is just silly when using Postgres (or any other db with transaction support in schema changes for that matter)

Didn’t understood why the issue is marked as resolved, even the initial question refers to error on version 1 but all answers just seem to ignore that detail. Maybe I’m missing something as well. The docs don’t seem to scratch this itch tough.

Some possible ways this lib could help the developer in tackling this corner case I can think of:

  • Allowing disabling the dirty feature
  • Adding method to drop schema_migrations table
  • Never setting first version as dirty (probably incoherent, bad idea)

All for helping out, just didn’t fully understood if this is something recognized as a problem.
I really (really) don’t want to sound like a jerk, but this bug makes the lib unusable in production. I cannot have an application that out of the blue requires a human messing with the DB.

Источник

Discussion on: How to write & run database migration in Golang


Collapse

Expand


acetorscode profile image

Getting this error while migratedown

«Dirty database version -1. Fix and force version.»

can anyone help with a solution! it will be a great help if you can.


Collapse

Expand


mquanit profile image

Mohammad Quanit

Hi 👋, I am Mohammad Quanit, and I’m a Product Engineering Manager at Atompoint. I have overall 5 years of experience in software engineering in which I’ve been mostly working on Javascript & Cloud

  • Email

  • Location

    Earth

  • Education

    Graduated in Software Engineering

  • Work

    Product Engineering Manager at Atompoint

  • Joined

    Jul 20, 2019

Jan 13 ’22

  • Copy link

Hi Apoorv, try this cmd first

migrate -path db/migration -database "postgresql://root:secret@localhost:5432/simple_bank?sslmode=disable" force 1

and then run makefile cmd

make migrateup

it should solve this issue. Here’s the docs for it
github.com/golang-migrate/migrate/…

For further actions, you may consider blocking this person and/or reporting abuse

How to Perform Database Migrations using Go Migrate

Since its introduction, the programming language Go (also known as Golang) has become increasingly popular. It is known for its simplicity and efficient performance, similar to that of a lower-level language such as C++.

While working with a database, schema migration is one of the most important tasks that developers do throughout the project lifecycle. In this article, I will explain what database migration is and how to manage it using go-migrate.

What is a Database Migration?

A database migration, also known as a schema migration, is a set of changes to be made to a structure of objects within a relational database.

It is a way to manage and implement incremental changes to the structure of data in a controlled, programmatic manner. These changes are often reversible, meaning they can be undone or rolled back if required.

The process of migration helps to change the database schema from its current state to a new desired state, whether it involves adding tables and columns, removing elements, splitting fields, or changing types and constraints.

By managing these changes in a programmatic way, it becomes easier to maintain consistency and accuracy in the database, as well as keep track of the history of modifications made to it.

Setup and Installation

migrate is a CLI tool that you can use to run migrations. You can easily install it on various operating systems such as Linux, Mac and Windows by using package managers like curl, brew, and scoop, respectively.

For more information on how to install and use the tool, you can refer to the official documentation.

To install the migrate CLI tool using scoop on Windows, you can follow these steps:

$ scoop install migrate

To install the migrate CLI tool using curl on Linux, you can follow these steps:

$ curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey| apt-key add -
$ echo "deb https://packagecloud.io/golang-migrate/migrate/ubuntu/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/migrate.list
$ apt-get update
$ apt-get install -y migrate

To install the migrate CLI tool using on Mac, you can follow these steps:

$ brew install golang-migrate

How to Create a New Migration

Create a directory like database/migration to store all the migration files.

image-263

Source files structure in GoLand IDE

Next, create migration files using the following command:

$ migrate create -ext sql -dir database/migration/ -seq init_mg

image-267

Terminal output displaying successful creation of migration

You use -seq to generate a sequential version and init_mg is the name of the migration.

image-269

Source files structure in GoLand IDE

A migration typically consists of two distinct files, one for moving the database to a new state (referred to as «up») and another for reverting the changes made to the previous state (referred to as «down»).

The «up» file is used to implement the desired changes to the database, while the «down» file is used to undo those changes and return the database to its previous state.

Database-migration

Flow of database migration

The format of those files for SQL are:

{version}_{title}.down.sql
{version}_{title}.up.sql

When you create migration files, they will be empty by default. To implement the changes you want, you will need to fill them with the appropriate SQL queries.

image-282

SQL queries for migrating data

How to Run Migration Up

In order to execute the SQL statements in the migration files, migrate requires a valid connection to a Postgres database.

To accomplish this, you will need to provide a connection string in the proper format.

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose up

Now in your Postgres shell, you can check newly created tables by using the following commands:

d+

d+ table_name DESCRIBE TABLE

image-286

Displaying table data in Postgres

How to Rollback Migrations

If you want to revert back the migration, you can do that using the following down tag:

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose down

It will delete the email column from both tables as mentioned in the 000002_init_mg.up.sql file.

Now, let’s check the database and see if email has been deleted or not:

Screenshot_20230126_102731

Displaying updated table data in Postgres

How to Resolve Migration Errors

If a migration contains an error and is executed, migrate will prevent any further migrations from being run on the same database.

An error message like Dirty database version 1. Fix and force version will be displayed, even after the error in the migration is fixed. This indicates that the database is «dirty» and needs to be investigated.

It is necessary to determine if the migration was applied partially or not at all. Once you’ve determined this, the database version should be forced to reflect its true state using the force command.

$ migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" force <VERSION>

How to Add Commands in a Makefile

migration_up: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose up

migration_down: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" -verbose down

migration_fix: migrate -path database/migration/ -database "postgresql://username:secretkey@localhost:5432/database_name?sslmode=disable" force VERSION

Now, you can run $ make migration_up for ‘up’, $ make migration_down for ‘down’, and $ make migration_fix to fix the migration issue.

Before running the makefile, ensure that the correct version number is included in the migration_fix command.

Conclusion

Migration systems typically generate files that can be shared across developers and multiple teams. They can also be applied to multiple databases and maintained in version control.

Keeping a record of changes to the database makes it possible to track the history of modifications made to it. This way, the database schema and the application’s understanding of that structure can evolve together.

That concludes our discussion on database migration. I hope you found the information to be useful and informative.

If you enjoyed reading this article, please consider sharing it with your colleagues and friends on social media. Additionally, please follow me on Twitter for more updates on technology and coding. Thank you for reading!



Learn to code for free. freeCodeCamp’s open source curriculum has helped more than 40,000 people get jobs as developers. Get started

@duyanghao Of course I can.

Let’s assume I want to create a new table. But I made a «slight» mistake (missing Primary Key).

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Now I try to migrate it:

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable up

migrate will refuse this:

error: 2 errors occurred:
	* migration failed: there is no primary key for referenced table "auth_users" in line 0: BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;
 (details: pq: there is no primary key for referenced table "auth_users")
	* pq: current transaction is aborted, commands ignored until end of transaction block in line 0: SELECT pg_advisory_unlock($1)

Now I fixed it and added the PRIMARY KEY to the migration:

BEGIN;

CREATE TABLE auth_users (
	id BIGSERIAL PRIMARY KEY,

	name VARCHAR(255),
	superior_id BIGINT REFERENCES auth_users
);

COMMIT;

If I try to migrate again, migrate will STILL refuse:

error: Dirty database version 16. Fix and force version.

So I have to go the last successfull version, which is 15.

migrate -path migrations/ -database postgres://test:test@localhost/dummy?sslmode=disable force 15

If i migrate again, the output will now be:

16/u create_auth_users (17.608619ms)

The same works if you want to go down. But if would be 17 in this case, not 15.
I hope this helps a bit.

Содержание

  1. Name already in use
  2. migrate / cmd / migrate / README.md
  3. Users who have contributed to this file
  4. Name already in use
  5. migrate / README.md
  6. golang-migrate/migrate
  7. Launching GitHub Desktop
  8. Launching GitHub Desktop
  9. Launching Xcode
  10. Launching Visual Studio Code
  11. Latest commit
  12. Git stats
  13. Files
  14. README.md
  15. Name already in use
  16. migrate / GETTING_STARTED.md
  17. How to write & run database migration in Golang
  18. Install golang-migrate
  19. Create a new migration
  20. Up/down migration
  21. Check postgres container status
  22. Access postgres container shell
  23. Create/drop database inside postgres container
  24. Create/drop database outside postgres container
  25. Write Makefile
  26. View database with TablePlus
  27. Run the migration
  28. Add migrate up/down to the Makefile

Name already in use

migrate / cmd / migrate / README.md

  • Go to file T
  • Go to line L
  • Copy path
  • Copy permalink

5 contributors

Users who have contributed to this file

Copy raw contents

Copy raw contents

Download pre-built binary (Windows, MacOS, or Linux)

Linux (*.deb package)

With Go toolchain

  1. Requires a version of Go that supports modules. e.g. Go 1.11+
  2. These examples build the cli which will only work with postgres. In order to build the cli for use with other databases, replace the postgres build tag with the appropriate database tag(s) for the databases desired. The tags correspond to the names of the sub-packages underneath the database package.
  3. Similarly to the database build tags, if you need to support other sources, use the appropriate build tag(s).
  4. Support for build constraints will be removed in the future: #60
  5. For versions of Go 1.15 and lower, make sure you’re not installing the migrate CLI from a module. e.g. there should not be any go.mod files in your current directory or any directory from your current directory to the root

So let’s say you want to run the first two migrations

If your migrations are hosted on github

The CLI will gracefully stop at a safe point when SIGINT (ctrl+c) is received. Send SIGKILL for immediate halt.

Reading CLI arguments from somewhere else

Источник

Name already in use

migrate / README.md

  • Go to file T
  • Go to line L
  • Copy path
  • Copy permalink

Copy raw contents

Copy raw contents

Database migrations written in Go. Use as CLI or import as library.

  • Migrate reads migrations from sources and applies them in correct order to a database.
  • Drivers are «dumb», migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
  • Database drivers don’t assume things or try to correct user input. When in doubt, fail.

Database drivers run migrations. Add a new database?

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: ! , # , $ , % , & , ‘ , ( , ) , * , + , , , / , : , ; , = , ? , @ , [ , ]

It’s easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

Source drivers read migrations from local or remote sources. Add a new source?

  • Filesystem — read from filesystem
  • io/fs — read from a Go io/fs
  • Go-Bindata — read from embedded binary data (jteeuwen/go-bindata)
  • pkger — read from embedded binary data (markbates/pkger)
  • GitHub — read from remote GitHub repositories
  • GitHub Enterprise — read from remote GitHub Enterprise repositories
  • Bitbucket — read from remote Bitbucket repositories
  • Gitlab — read from remote Gitlab repositories
  • AWS S3 — read from Amazon Web Services S3
  • Google Cloud Storage — read from Google Cloud Platform Storage

Источник

golang-migrate/migrate

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Database migrations written in Go. Use as CLI or import as library.

  • Migrate reads migrations from sources and applies them in correct order to a database.
  • Drivers are «dumb», migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.)
  • Database drivers don’t assume things or try to correct user input. When in doubt, fail.

Database drivers run migrations. Add a new database?

Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: dbdriver://username:password@host:port/dbname?param1=true&param2=false

Any reserved URL characters need to be escaped. Note, the % character also needs to be escaped

Explicitly, the following characters need to be escaped: ! , # , $ , % , & , ‘ , ( , ) , * , + , , , / , : , ; , = , ? , @ , [ , ]

It’s easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below:

Source drivers read migrations from local or remote sources. Add a new source?

  • Filesystem — read from filesystem
  • io/fs — read from a Go io/fs
  • Go-Bindata — read from embedded binary data (jteeuwen/go-bindata)
  • pkger — read from embedded binary data (markbates/pkger)
  • GitHub — read from remote GitHub repositories
  • GitHub Enterprise — read from remote GitHub Enterprise repositories
  • Bitbucket — read from remote Bitbucket repositories
  • Gitlab — read from remote Gitlab repositories
  • AWS S3 — read from Amazon Web Services S3
  • Google Cloud Storage — read from Google Cloud Platform Storage

Источник

Name already in use

migrate / GETTING_STARTED.md

  • Go to file T
  • Go to line L
  • Copy path
  • Copy permalink

Copy raw contents

Copy raw contents

Before you start, you should understand the concept of forward/up and reverse/down database migrations.

Configure a database for your application. Make sure that your database driver is supported here.

Create some migrations using migrate CLI. Here is an example:

Once you create your files, you should fill them.

IMPORTANT: In a project developed by more than one person there is a chance of migrations inconsistency — e.g. two developers can create conflicting migrations, and the developer that created their migration later gets it merged to the repository first. Developers and Teams should keep an eye on such cases (especially during code review). Here is the issue summary if you would like to read more.

Consider making your migrations idempotent — we can run the same sql code twice in a row with the same result. This makes our migrations more robust. On the other hand, it causes slightly less control over database schema — e.g. let’s say you forgot to drop the table in down migration. You run down migration — the table is still there. When you run up migration again — CREATE TABLE would return an error, helping you find an issue in down migration, while CREATE TABLE IF NOT EXISTS would not. Use those conditions wisely.

In case you would like to run several commands/queries in one migration, you should wrap them in a transaction (if your database supports it). This way if one of commands fails, our database will remain unchanged.

Run your migrations through the CLI or your app and check if they applied expected changes. Just to give you an idea:

Just add the code to your app and you’re ready to go!

Before commiting your migrations you should run your migrations up, down, and then up again to see if migrations are working properly both ways. (e.g. if you created a table in a migration but reverse migration did not delete it, you will encounter an error when running the forward migration again) It’s also worth checking your migrations in a separate, containerized environment. You can find some tools at the end of this document.

IMPORTANT: If you would like to run multiple instances of your app on different machines be sure to use a database that supports locking when running migrations. Otherwise you may encounter issues.

Forcing your database version

In case you run a migration that contained an error, migrate will not let you run other migrations on the same database. You will see an error like Dirty database version 1. Fix and force version , even when you fix the erred migration. This means your database was marked as ‘dirty’. You need to investigate the migration error — was your migration applied partially, or was it not applied at all? Once you know, you should force your database to a version reflecting it’s real state. You can do so with force command:

Once you force the version and your migration was fixed, your database is ‘clean’ again and you can proceed with your migrations.

For details and example of usage see this comment.

Источник

How to write & run database migration in Golang

When working with database, schema migration is one important task that we often have to do throughout the application lifetime to adapt to new business requirements.

In this lecture, we will learn how to write and run database schema migration in Golang using golang-migrate library.

Install golang-migrate

Golang-migrate works with many different database engines like postgres, mysql, mongo, cockroach, etc.

Let’s open this CLI documentation to see how to install it. I’m on a mac, so I will use Homebrew.

Exit fullscreen mode

Migrate gives us several commands:

  • The first one is create , which we can use to create new migration files.
  • The 2nd one is goto , which will migrate the schema to a specific version.
  • Then the up or down commands to apply all or N up or down migrations.

There are several more commands, but in most of the time, we will work with create , up , and down .

Alright, migrate is successfully installed. We can run migrate -help to read its manual.

Create a new migration

OK, now I’m gonna create a new folder for our Simple Bank project. And inside, I will create a new folder db/migration to store all of our migration files.

Exit fullscreen mode

Then let’s create the 1st migration file to initialise our Simple Bank’s database schema.

Start with migrate create . Then the extension of the file will be sql , and the directory to store it is db/migration .

Exit fullscreen mode

We use the -seq flag to generate a sequential version number for the migration file. And finally the name of the migration, which is init_schema in this case.

As you can see, 2 migration files have been generated for us. They both have version 1 in the file name’s prefix, but their suffixes are different: 1 file is up and the other is down . Why?

Up/down migration

Well, basically it’s a best practice when writing database migration. The up script is run to make a forward change to the schema. And the down script is run if we want to revert the change made by the up script.

So when we run migrate up command, The up script files inside db/migration folder will be run sequentially by the order of their prefix version.

On the contrary, when we run migrate down command, the down-script files inside db/migration folder will be run sequentially by the reverse order of their prefix version.

Aright, now let’s open the simple_bank.sql file that we’ve generated in previous lectures. I’m gonna copy all content of this file and paste it to the init_schema.up.sql file.

Exit fullscreen mode

For the init_schema.down.sql file, we should revert the changes made by the up script. In this case, the up script creates 3 tables: accounts , transfers , and entries . So the down script should remove all of them. We use the DROP TABLE query for this purpose.

Exit fullscreen mode

Here we drop entries and transfers table before dropping the accounts table because there’s a foreign key constraint in entries and transfers that references accounts records.

OK, so now our migration scripts are ready. Let’s try to run them.

Check postgres container status

But before that, we should check if our postgres container is still running or not:

Exit fullscreen mode

By the way, I’m gonna show you some more docker commands to work with containers. If we want to stop a running container, we use docker stop with the container name or ID.

Exit fullscreen mode

After this, if we run docker ps , we won’t see the postgres container anymore because it’s not running. To list all containers, regardless of their running status, we can run:

Exit fullscreen mode

Now we see our postgres container with status exited .

To turn it back on, We just need to run: docker start and pass in the container name or ID.

Exit fullscreen mode

Then here we go, the postgres12 container is now up and running.

Access postgres container shell

We can access its shell with the docker exec command. As we’re using postgres alpine image, we don’t have /bin/bash shell as in ubuntu, so we use /bin/sh shell instead:

Exit fullscreen mode

Inside the shell, we have access to all standard linux commands.

And since this is a postgres container, it also gives us some CLI commands to interact with postgres server directly from the shell.

Create/drop database inside postgres container

We have to create the database before we can run the 1st migration.

So let’s run the createdb command inside the postgres container’s shell to create a new database for our Simple Bank:

Exit fullscreen mode

  • We use the —username option to say that we’re connecting as root user.
  • And the —owner option to say that the database we’re going to create will belong to root user as well.
  • The last argument is the database name: simple_bank .

OK, the database is created. We can access its console with the psql command.

We can also delete the database using dropdb command and pass in the name of the database.

Exit fullscreen mode

We use exit command to get out of the container shell.

Create/drop database outside postgres container

Now from outside of the container, we can also run createdb directly with the docker exec command.

Exit fullscreen mode

And access the database console without going through the container shell.

Exit fullscreen mode

Write Makefile

OK, now I’m gonna create a Makefile in our project, then add a createdb command to create the simple bank database, and a dropdb command to delete it.

When working in a team, these commands will be useful for your team mates to easily setup the project on their local machine for development.

Let’s add the command that we used to start postgres container in the previous lecture to the Makefile as well.

Exit fullscreen mode

Alright, Now let’s stop the current postgres container.

Exit fullscreen mode

The container is stopped. I’m gonna remove it completely using docker rm command.

Exit fullscreen mode

Now when we run

Exit fullscreen mode

A new postgres container will be started. And we can run

Exit fullscreen mode

to create the simple_bank database.

View database with TablePlus

OK, the database is created. Let’s connect to it using TablePlus.

The connection that we’ve setup in the previous lecture will bring us to the root database. We can click on the database icon to open our new simple_bank database.

Alright, now you can see 2 databases here: root and simple_bank . For now the simple_bank database is empty. So let’s go back to the terminal and run the first migration.

Run the migration

Start with migrate . Then we use the -path option to specify the folder contains our migration files, which is db/migration .

Exit fullscreen mode

The -database option is used to specify the URL to the database server.

  • We’re using postgres, so the driver name is postgresql .
  • Then the username is root
  • The password is secret
  • The address is localhost , port 5432 .
  • And the database name is simple_bank .

We use -verbose option to ask migrate to print verbose logging.

And finally we use the up argument to tell migrate to run migrate up command.

Oh, we’ve got an error: SSL is not enabled on the server. That’s because our postgres container doesn’t enable SSL by default.

So we should add sslmode=disable parameter to the database URL. Now run this command:

Exit fullscreen mode

And the migration is successful!

If we refresh the simple bank database in TablePlus, we can now see 4 tables: accounts , entries , transfers , and schema_migrations .

The schema_migrations table stores the latest applied migration version, which in our case is version 1, because we have run only 1 single migration file.

The dirty column tells us if the last migration has failed or not. If it fails, we must manually fix the issues to make the database state clean before trying to run any other migration versions.

Add migrate up/down to the Makefile

OK, now I’m gonna add the migrate up and migrate down command to the Makefile:

Exit fullscreen mode

Now let’s try them in the terminal! First I will run:

Exit fullscreen mode

And go back to TablePlus and refresh.

All tables are gone, except for the schema_migrations table.

OK, now let’s run:

Exit fullscreen mode

Then refresh TablePlus.

All tables are back again. Excellent!

So that wraps up today’s lecture about database migration. Thank you for reading and see you in the next lecture!

If you like the article, please subscribe to our Youtube channel and follow us on Twitter for more tutorials in the future.

If you want to join me on my current amazing team at Voodoo, check out our job openings here. Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.

Источник

Могут возникнуть таймауты, и правильное решение неочевидно, поэтому необходимо вмешательство пользователя.
например, миграция никогда не могла применяться, частично или полностью.

@dhui Как исправить Dirty database status ? Для этого есть какие-то документы?

@duyanghao Перейти к предыдущей версии, а затем снова запустить «вверх».

Поэтому, если вы получите ошибку в версии 10, исправьте ошибку и затем принудительно установите версию 9. После этого вы можете выполнить миграцию снова.

@duyanghao Перейти к предыдущей версии, а затем снова запустить «вверх».

Поэтому, если вы получите ошибку в версии 10, исправьте ошибку и затем принудительно установите версию 9. После этого вы можете выполнить миграцию снова.

@anihex Не могли бы вы привести пример исправления Dirty database status как описано выше?

@duyanghao Конечно, могу.

Предположим, я хочу создать новую таблицу. Но я сделал «небольшую» ошибку (пропустил первичный ключ).

BEGIN;

CREATE TABLE auth_users (
    id BIGSERIAL,

    name VARCHAR(255),
    superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Теперь пытаюсь перенести:

migrate -path migrations/ -database postgres://test:[email protected]/dummy?sslmode=disable up

migrate откажется от этого:

error: 2 errors occurred:
    * migration failed: there is no primary key for referenced table "auth_users" in line 0: BEGIN;

CREATE TABLE auth_users (
    id BIGSERIAL,

    name VARCHAR(255),
    superior_id BIGINT REFERENCES auth_users
);

COMMIT;
 (details: pq: there is no primary key for referenced table "auth_users")
    * pq: current transaction is aborted, commands ignored until end of transaction block in line 0: SELECT pg_advisory_unlock($1)

Теперь я исправил это и добавил ПЕРВИЧНЫЙ КЛЮЧ к миграции:

BEGIN;

CREATE TABLE auth_users (
    id BIGSERIAL PRIMARY KEY,

    name VARCHAR(255),
    superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Если я попытаюсь выполнить миграцию еще раз, миграция все равно будет отказываться:

error: Dirty database version 16. Fix and force version.

Так что мне нужно использовать последнюю успешную версию — 15.

migrate -path migrations/ -database postgres://test:[email protected]/dummy?sslmode=disable force 15

Если я перейду снова, результат будет следующим:

16/u create_auth_users (17.608619ms)

То же самое работает, если вы хотите спуститься. Но если бы в данном случае было 17, а не 15.
Надеюсь, это немного поможет.

@duyanghao Конечно, могу.

Предположим, я хочу создать новую таблицу. Но я сделал «небольшую» ошибку (пропустил первичный ключ).

BEGIN;

CREATE TABLE auth_users (
  id BIGSERIAL,

  name VARCHAR(255),
  superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Теперь пытаюсь перенести:

migrate -path migrations/ -database postgres://test:[email protected]/dummy?sslmode=disable up

migrate откажется от этого:

error: 2 errors occurred:
  * migration failed: there is no primary key for referenced table "auth_users" in line 0: BEGIN;

CREATE TABLE auth_users (
  id BIGSERIAL,

  name VARCHAR(255),
  superior_id BIGINT REFERENCES auth_users
);

COMMIT;
 (details: pq: there is no primary key for referenced table "auth_users")
  * pq: current transaction is aborted, commands ignored until end of transaction block in line 0: SELECT pg_advisory_unlock($1)

Теперь я исправил это и добавил ПЕРВИЧНЫЙ КЛЮЧ к миграции:

BEGIN;

CREATE TABLE auth_users (
  id BIGSERIAL PRIMARY KEY,

  name VARCHAR(255),
  superior_id BIGINT REFERENCES auth_users
);

COMMIT;

Если я попытаюсь выполнить миграцию еще раз, миграция все равно будет отказываться:

error: Dirty database version 16. Fix and force version.

Так что мне нужно использовать последнюю успешную версию — 15.

migrate -path migrations/ -database postgres://test:[email protected]/dummy?sslmode=disable force 15

Если я перейду снова, результат будет следующим:

16/u create_auth_users (17.608619ms)

То же самое работает, если вы хотите спуститься. Но если бы в данном случае было 17, а не 15.
Надеюсь, это немного поможет.

@anihex Спасибо за ваш ответ, это действительно очень помогает.

Может, я что-то здесь упускаю. Если мы выполняем миграции внутри транзакции, зачем нам помечать базу данных как грязную? Спрашивать об этом, потому что это замедляет быстрые итерации миграции, если разработчик должен форсировать после каждого сбоя, и когда это самая первая миграция, вы не можете принудительно перейти к версии 000000

@ bkakadiya42 У меня была такая же проблема, в итоге я

Это помогает решить проблему, но при этом остается открытым вопрос, почему неудачная транзакция загрязняет базу данных. Это должно быть ошибка. «Силовой» UX также очень враждебен пользователям.

Может, я что-то здесь упускаю. Если мы выполняем миграции внутри транзакции, зачем нам помечать базу данных как грязную? Спрашивать об этом, потому что это замедляет быстрые итерации миграции, если разработчик должен форсировать после каждого сбоя, и когда это самая первая миграция, вы не можете принудительно перейти к версии 000000

Пятно на. Ошибка при первой миграции кажется неисправимой через интерфейс migrate . Я взломаю и отброшу таблицу schema_migrations (если грязная && Version == 1), но это явно нарушение абстракции.
Кроме того, все это просто глупо при использовании Postgres (или любого другого db с поддержкой транзакций в изменениях схемы, если на то пошло)

Не понял, почему проблема помечена как решенная, даже первоначальный вопрос относится к ошибке в версии 1, но все ответы, похоже, просто игнорируют эту деталь. Может, я тоже что-то упускаю. Документы, похоже, не решают этого зуда.

Некоторые возможные способы, которыми эта библиотека может помочь разработчику в решении этого углового случая, о котором я могу думать:

  • Разрешение отключения грязной функции
  • Добавление метода для удаления таблицы schema_migrations
  • Никогда не устанавливайте первую версию как грязную (возможно, бессвязная, плохая идея)

Все для помощи, просто не совсем понял, если это что-то признано проблемой.
Я действительно (действительно) не хочу показаться придурком, но эта ошибка делает библиотеку непригодной для использования в продакшене. У меня не может быть приложения, которое неожиданно требует вмешательства человека в БД.

Cheerio!

Я нажимаю на это при развертывании гавани с оператором куба.

2020-09-03T14:39:47Z [INFO] [/common/dao/pgsql.go:127]: Upgrading schema for pgsql ...
2020-09-03T14:39:47Z [ERROR] [/common/dao/pgsql.go:132]: Failed to upgrade schema, error: "Dirty database version 10. Fix and force version."
2020-09-03T14:39:47Z [FATAL] [/core/main.go:123]: failed to migrate: Dirty database version 10. Fix and force version.

Если мы выполняем миграции внутри транзакции, зачем нам помечать базу данных как грязную?

по-прежнему остается открытым вопрос, почему неудачная транзакция делает базу данных грязной

migrate не обеспечивает этого (например, не выполняет все миграции в транзакции), поэтому он не может автоматически откатиться в случае сбоя. Если все ваши миграции заключены в транзакцию, вы можете безопасно выполнить миграцию к предыдущей примененной миграции. В противном случае вам нужно будет проверить состояние вашей базы данных и определить, что делать дальше.

«Силовой» UX также очень враждебен пользователям.

Даже если migrate удалось откатить неудачную миграцию, скорее всего, потребуется вмешательство пользователя, чтобы исправить какое-то состояние для успешной миграции. например, повторный запуск той же миграции с тем же начальным состоянием не удастся
Сообщать разработчику / системному администратору о том, что что-то сломано и требует их внимания, и заставить их признать, что проблема решена, — это хорошо. Однако я согласен с тем, что пользовательский интерфейс мог бы быть более плавным. например, принуждение — это дополнительный шаг, а не просто попытка повторно применить миграцию

@nnachefski Похоже на проблему с гаванью, пожалуйста, найдите / отправьте туда проблему.
FWIW, вот их миграции

Была ли эта страница полезной?

0 / 5 — 0 рейтинги

Понравилась статья? Поделить с друзьями:
  • Error dirty cow exploit failed no ta backup done
  • Error directory migrations already exists and is not empty
  • Error directory is not installable neither setup py nor pyproject toml found
  • Error direction closed or blacklist перевод
  • Error direct3d e invalidarg