Task #8569
Updated by Amy Forrester over 6 years ago
**from Tom Saleeba:**
I've decided to update both MNs to that latest version, 2.4.2, but I've hit a road block that I'm hoping you can help with. I ran the pip --upgrade dataone.gmn no worries and I've created the settings.py file in the site-packages/d1_gmn/ dir (we were on 2.0.5 which used site-packages/gmn/) so I can start the server and see the node description document. When I try to list the objects in the server I get the following error:
2018-04-20 03:14:05 ERROR django.request exception 24133 140418821256960 Internal Server Error: /mn/v2/object
Traceback (most recent call last):
...see attach file
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 131, in get_response
response = middleware_method(request, response)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 56, in process_response
response = self._serialize_object(request, view_result)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 102, in _serialize_object
request, view_result['query'], view_result['start'], view_result['total']
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 117, in _generate_object_list
for complete error/message row in db_query:
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 250, in __iter__
self._fetch_all()
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1118, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
raise original_exception
ProgrammingError: column app_scienceobject.filename does not exist
LINE 1: ...aded_timestamp", "app_scienceobject"."format_id", "app_scien...
Ahh, we just need a database migration as per https://dataone-python.readthedocs.io/en/latest/gmn/maintenance.html#upgrading-the-gmn-2-x-software-stack-to-the-latest-release. So I run `python manage.py migrate` but then...
Operations to perform:
Apply all migrations: app, auth, contenttypes
Running migrations:
Applying app.0001_initial...Traceback (most recent call last):
File "manage.py", line 31, in <module>
execute_from_command_line(sys.argv)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
fake_initial=fake_initial,
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
state = migration.apply(state, schema_editor)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 97, in database_forwards
schema_editor.create_model(model)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 303, in create_model
self.execute(sql, params or None)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 120, in execute
cursor.execute(sql, params)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "app_blockedmembernode" already exists
When I look in the Postgres DB, the "django_migrations" table has no rows. Is that right? I feel like I stuffed something up earlier in the life of this instance and now Django is complaining because it think it's starting with a fresh database so it should be empty. I expect that if it knew it was just upgrading between versions, it would deal with existing tables.
I can create a fresh database and apply the migration (I tried it) but then all the data is lost. I've also tried renaming that app_blockedmembernode table but then it complains about the next table app_event so it's probably going to complain about all of them. I was even cheeky and tried `...migrate --fake-initial` but that still complains the app_blockedmembernode exists, no change.
Hoping you can impart some wisdom. And that the wisdom isn't "compare the migrations with your schema by hand to figure out where you are, make the required changes and then force the migration with --fake" because that sounds tedious.
I've decided to update both MNs to that latest version, 2.4.2, but I've hit a road block that I'm hoping you can help with. I ran the pip --upgrade dataone.gmn no worries and I've created the settings.py file in the site-packages/d1_gmn/ dir (we were on 2.0.5 which used site-packages/gmn/) so I can start the server and see the node description document. When I try to list the objects in the server I get the following error:
2018-04-20 03:14:05 ERROR django.request exception 24133 140418821256960 Internal Server Error: /mn/v2/object
Traceback (most recent call last):
...see attach file
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 131, in get_response
response = middleware_method(request, response)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 56, in process_response
response = self._serialize_object(request, view_result)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 102, in _serialize_object
request, view_result['query'], view_result['start'], view_result['total']
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/d1_gmn/app/middleware/response_handler.py", line 117, in _generate_object_list
for complete error/message row in db_query:
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 250, in __iter__
self._fetch_all()
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1118, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
raise original_exception
ProgrammingError: column app_scienceobject.filename does not exist
LINE 1: ...aded_timestamp", "app_scienceobject"."format_id", "app_scien...
Ahh, we just need a database migration as per https://dataone-python.readthedocs.io/en/latest/gmn/maintenance.html#upgrading-the-gmn-2-x-software-stack-to-the-latest-release. So I run `python manage.py migrate` but then...
Operations to perform:
Apply all migrations: app, auth, contenttypes
Running migrations:
Applying app.0001_initial...Traceback (most recent call last):
File "manage.py", line 31, in <module>
execute_from_command_line(sys.argv)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
utility.execute()
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 204, in handle
fake_initial=fake_initial,
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 115, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
state = migration.apply(state, schema_editor)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 129, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 97, in database_forwards
schema_editor.create_model(model)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 303, in create_model
self.execute(sql, params or None)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 120, in execute
cursor.execute(sql, params)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/var/local/dataone/gmn_venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "app_blockedmembernode" already exists
When I look in the Postgres DB, the "django_migrations" table has no rows. Is that right? I feel like I stuffed something up earlier in the life of this instance and now Django is complaining because it think it's starting with a fresh database so it should be empty. I expect that if it knew it was just upgrading between versions, it would deal with existing tables.
I can create a fresh database and apply the migration (I tried it) but then all the data is lost. I've also tried renaming that app_blockedmembernode table but then it complains about the next table app_event so it's probably going to complain about all of them. I was even cheeky and tried `...migrate --fake-initial` but that still complains the app_blockedmembernode exists, no change.
Hoping you can impart some wisdom. And that the wisdom isn't "compare the migrations with your schema by hand to figure out where you are, make the required changes and then force the migration with --fake" because that sounds tedious.
