diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 9968a6348bae142cc15b8b22895cdb4bb1a93393..68308980adcacc29d678c5ec8e33bf121e4ec2d8 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -14,6 +14,8 @@ the setting is not present, the API is disabled). LMS: Added endpoints for AJAX requests to enable/disable notifications (which are not yet implemented) and a one-click unsubscribe page. +Common: Add a manage.py that knows about edx-platform specific settings and projects + Common: Added *experimental* support for jsinput type. Common: Added setting to specify Celery Broker vhost diff --git a/README.md b/README.md index efa2c78640692472f32a526fe01384c47c750443..4ab07b355022496354afa771f5ed8c27b164eef6 100644 --- a/README.md +++ b/README.md @@ -15,14 +15,14 @@ installation process. 3. Install VirtualBox: https://www.virtualbox.org/wiki/Downloads See http://docs.vagrantup.com/v2/providers/index.html for a list of supported Providers. You should use VirtualBox >= 4.2.12. - (Windows: later/earlier VirtualBox versions than 4.2.12 have been reported to not work well with + (Windows: later/earlier VirtualBox versions than 4.2.12 have been reported to not work well with Vagrant. If this is still a problem, you can install 4.2.12 from https://www.virtualbox.org/wiki/Download_Old_Builds_4_2). 4. Install Vagrant: http://www.vagrantup.com/ (Vagrant 1.2.2 or later) 5. Open a terminal 6. Download the project: `git clone git://github.com/edx/edx-platform.git` 7. Enter the project directory: `cd edx-platform/` -8. (Windows only) Run the commands to +8. (Windows only) Run the commands to [deal with line endings and symlinks under Windows](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#dealing-with-line-endings-and-symlinks-under-windows) 9. Create the development environment and start it: `vagrant up` @@ -33,7 +33,7 @@ install dependencies and configure the VM. This will take a while; go grab a coffee. When complete, you should see a _"Success!"_ message. -If not, refer to the +If not, refer to the [troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting). Your development environment is initialized only on the first bring-up. @@ -62,7 +62,7 @@ Using edX When you login to your VM, you are in `/opt/edx/edx-platform` by default, which is shared from your host workspace. Your host computer contains the edx-project development code and repository. -Your VM runs edx-platform code mounted from your host, so +Your VM runs edx-platform code mounted from your host, so you can develop by editing on your host. After logging into your VM with `vagrant ssh`, @@ -146,7 +146,7 @@ You should now also see a login on [http://localhost:6080/admin/]() You will need a privileged user for the admin login. You can create a CMS/LMS super-user with: ``` -$ rake django-admin["createsuperuser"] +$ ./manage.py lms createsuperuser ``` @@ -185,7 +185,7 @@ $ vagrant up # will make a new VM Troubleshooting --------------- -If anything doesn't work as expected, see the +If anything doesn't work as expected, see the [troubleshooting section](https://github.com/edx/edx-platform/wiki/Simplified-install-with-vagrant#troubleshooting). Installation - Advanced @@ -293,23 +293,12 @@ or any other process management tool. Configuring Your Project ------------------------ -We use [`rake`](http://rake.rubyforge.org/) to execute common tasks in our -project. The `rake` tasks are defined in the `rakefile`, or you can run `rake -T` -to view a summary. - Before you run your project, you need to create a sqlite database, create -tables in that database, run database migrations, and populate templates for -CMS templates. Fortunately, `rake` will do all of this for you! Just run: - - $ rake django-admin[syncdb] - $ rake django-admin[migrate] +tables in that database, and run database migrations. Fortunately, `django` +will do all of this for you -If you are running these commands using the [`zsh`](http://www.zsh.org/) shell, -zsh will assume that you are doing -[shell globbing](https://en.wikipedia.org/wiki/Glob_%28programming%29), search for -a file in your directory named `django-adminsyncdb` or `django-adminmigrate`, -and fail. To fix this, just surround the argument with quotation marks, so that -you're running `rake "django-admin[syncdb]"`. + $ ./manage.py lms syncdb --migrate + $ ./manage.py cms syncdb --migrate Run Your Project ---------------- @@ -317,6 +306,10 @@ edX has two components: Studio, the course authoring system; and the LMS (learning management system) used by students. These two systems communicate through the MongoDB database, which stores course information. +We use [`rake`](http://rake.rubyforge.org/) to execute common tasks in our +project. The `rake` tasks are defined in the `rakefile`, or you can run `rake -T` +to view a summary. + To run Studio, run: $ rake cms diff --git a/cms/manage.py b/cms/manage.py deleted file mode 100755 index 723fa59da17cbf14ed55f189a3c37a8bad65e601..0000000000000000000000000000000000000000 --- a/cms/manage.py +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env python -from django.core.management import execute_manager -import imp -try: - imp.find_module('settings') # Assumed to be in the same directory. -except ImportError: - import sys - sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. " - "It appears you've customized things.\nYou'll have to run django-admin.py, " - "passing it your settings module.\n" % __file__) - sys.exit(1) - -import settings - -if __name__ == "__main__": - execute_manager(settings) diff --git a/common/djangoapps/external_auth/models.py b/common/djangoapps/external_auth/models.py index 6c2f38d8b31cd47dba119374e53ea7bfd1e9cfc6..b40e4274343b5b8edb2e60c33d711cb13d139e9c 100644 --- a/common/djangoapps/external_auth/models.py +++ b/common/djangoapps/external_auth/models.py @@ -4,9 +4,9 @@ WE'RE USING MIGRATIONS! If you make changes to this model, be sure to create an appropriate migration file and check it in at the same time as your model changes. To do that, -1. Go to the mitx dir -2. django-admin.py schemamigration student --auto --settings=lms.envs.dev --pythonpath=. description_of_your_change -3. Add the migration file created in mitx/common/djangoapps/external_auth/migrations/ +1. Go to the edx-platform dir +2. ./manage.py lms schemamigration student --auto description_of_your_change +3. Add the migration file created in edx-platform/common/djangoapps/external_auth/migrations/ """ from django.db import models diff --git a/common/djangoapps/student/management/commands/pearson_transfer.py b/common/djangoapps/student/management/commands/pearson_transfer.py index 75716c744370a173faa52ed312dd0625f3b0ef5d..39e1e7468e73987ba83b610b80076706ee829289 100644 --- a/common/djangoapps/student/management/commands/pearson_transfer.py +++ b/common/djangoapps/student/management/commands/pearson_transfer.py @@ -20,7 +20,7 @@ class Command(BaseCommand): files and then uploads over SFTP to Pearson and stuffs the entry in an S3 bucket for archive purposes. - Usage: django-admin.py pearson-transfer --mode [import|export|both] + Usage: ./manage.py pearson-transfer --mode [import|export|both] """ option_list = BaseCommand.option_list + ( diff --git a/common/djangoapps/student/models.py b/common/djangoapps/student/models.py index e6530338a8585d8cf7286018f2a97026a6df5850..97d7e8b0289a85e908a308ee166a0816d2b440d5 100644 --- a/common/djangoapps/student/models.py +++ b/common/djangoapps/student/models.py @@ -6,9 +6,9 @@ Migration Notes If you make changes to this model, be sure to create an appropriate migration file and check it in at the same time as your model changes. To do that, -1. Go to the mitx dir -2. django-admin.py schemamigration student --auto --settings=lms.envs.dev --pythonpath=. description_of_your_change -3. Add the migration file created in mitx/common/djangoapps/student/migrations/ +1. Go to the edx-platform dir +2. ./manage.py lms schemamigration student --auto description_of_your_change +3. Add the migration file created in edx-platform/common/djangoapps/student/migrations/ """ from datetime import datetime import hashlib diff --git a/doc/development.md b/doc/development.md index e6ab650002ddb255c64cff98d66501c66fd9f1cd..5b0e606adbe9bc871362d671d44fff11b3db5a29 100644 --- a/doc/development.md +++ b/doc/development.md @@ -48,14 +48,14 @@ Both the LMS and Studio can be started using the following shortcut tasks rake lms[cms.dev] # Start LMS to run alongside Studio rake lms[cms.dev_preview] # Start LMS to run alongside Studio in preview mode -Under the hood, this executes `django-admin.py runserver --pythonpath=$WORKING_DIRECTORY --settings=lms.envs.dev`, +Under the hood, this executes `./manage.py {lms|cms} --settings $ENV runserver`, which starts a local development server. Both of these commands take arguments to start the servers in different environments or with additional options: # Start the LMS using the test configuration, on port 5000 - rake lms[test,5000] # Executes django-admin.py runserver --pythonpath=$WORKING_DIRECTORY --setings=lms.envs.test 5000 + rake lms[test,5000] # Executes ./manage.py lms --settings test runserver 5000 *N.B.* You may have to escape the `[` characters, depending on your shell: `rake "lms[test,5000]"` diff --git a/doc/discussion.md b/doc/discussion.md index 752dc6a5e7dab88261939a0dea6356ffc2d40f06..12c139c62d782baf2f4aa229cd0faa42f052755d 100644 --- a/doc/discussion.md +++ b/doc/discussion.md @@ -68,24 +68,19 @@ First make sure that the database is up-to-date: If you have created users in the edx-platform django apps when the comment service was not running, you will need to one-way sync the users into the comment service back end database: - rake django-admin[sync_user_info] - -For convenience, add the following environment variables to the terminal (assuming that you're using configuration set lms.envs.dev): - - export DJANGO_SETTINGS_MODULE=lms.envs.dev - export PYTHONPATH=. + ./manage.py lms sync_user_info Now initialize roles and permissions, providing a course id. See the example below. Note that you do not need to do this for Studio-created courses, as the Studio application does this for you. - django-admin.py seed_permissions_roles "MITx/6.002x/2012_Fall" + ./manage.py lms seed_permissions_roles "MITx/6.002x/2012_Fall" To assign yourself as a moderator, use the following command (assuming your username is "test", and the course id is "MITx/6.002x/2012_Fall"): - django-admin.py assign_role test Moderator "MITx/6.002x/2012_Fall" + ./manage.py lms assign_role test Moderator "MITx/6.002x/2012_Fall" To assign yourself as an administrator, use the following command - django-admin.py assign_role test Administrator "MITx/6.002x/2012_Fall" + ./manage.py lms assign_role test Administrator "MITx/6.002x/2012_Fall" ## Some other useful commands @@ -155,8 +150,8 @@ You can use the following command to launch a console within the service environ Use the following command to see the roles and permissions of a user in a given course (assuming, again, that the username is "test"): - django-admin.py show_permissions moderator + ./manage.py lms show_permissions moderator You need to make sure that the environment variables are exported. Otherwise you would need to do - django-admin.py show_permissions moderator --settings=lms.envs.dev --pythonpath=. + ./manage.py lms show_permissions moderator diff --git a/doc/testing.md b/doc/testing.md index 96f0dcfd98817029d1c4b8f775aad7ae7a57e848..cfca297f2b5b1327d5c3ae8c1936011fbb5c0ebe 100644 --- a/doc/testing.md +++ b/doc/testing.md @@ -233,7 +233,7 @@ generates HTML and XML (Cobertura format) reports. When testing problems that use a queue server on AWS (e.g. sandbox-xqueue.edx.org), you'll need to run your server on your public IP, like so. -`django-admin.py runserver --settings=lms.envs.dev --pythonpath=. 0.0.0.0:8000` +`./manage.py lms runserver 0.0.0.0:8000` When you connect to the LMS, you need to use the public ip. Use `ifconfig` to figure out the number, and connect e.g. to `http://18.3.4.5:8000/` diff --git a/lms/djangoapps/courseware/tests/test_masquerade.py b/lms/djangoapps/courseware/tests/test_masquerade.py index 3dc3d2b6b1b6c193912a6c59569afc2e25208c9b..0fc4eae242a25231fb862b6d3bb77a1419284a79 100644 --- a/lms/djangoapps/courseware/tests/test_masquerade.py +++ b/lms/djangoapps/courseware/tests/test_masquerade.py @@ -5,7 +5,7 @@ Based on (and depends on) unit tests for courseware. Notes for running by hand: -django-admin.py test --settings=lms.envs.test --pythonpath=. lms/djangoapps/courseware +./manage.py lms --settings test test lms/djangoapps/courseware """ from django.test.utils import override_settings diff --git a/lms/djangoapps/instructor/tests/test_download_csv.py b/lms/djangoapps/instructor/tests/test_download_csv.py index fd5bd562badd81e36bf6994572dff4c6d59e439d..5a934d7a2319898fbca6aeb3e738f3d77543534a 100644 --- a/lms/djangoapps/instructor/tests/test_download_csv.py +++ b/lms/djangoapps/instructor/tests/test_download_csv.py @@ -5,7 +5,7 @@ Based on (and depends on) unit tests for courseware. Notes for running by hand: -django-admin.py test --settings=lms.envs.test --pythonpath=. lms/djangoapps/instructor +./manage.py lms --settings test test lms/djangoapps/instructor """ from django.test.utils import override_settings diff --git a/lms/djangoapps/notes/README.md b/lms/djangoapps/notes/README.md index 2e81fa5ec1680cad986f687a2f27af17f16a0584..233575ddacf914f252ba008fe2c3c79e413151c2 100644 --- a/lms/djangoapps/notes/README.md +++ b/lms/djangoapps/notes/README.md @@ -30,8 +30,7 @@ Developer Overview ### Quickstart ``` -$ rake django-admin[syncdb] -$ rake django-admin[migrate] +$ ./manage.py lms syncdb --migrate ``` Then follow the steps above to enable the *My Notes* tab or manually add a tab to the policy tab configuration with ```{"type": "notes", "name": "My Notes"}```. diff --git a/lms/djangoapps/open_ended_grading/tests.py b/lms/djangoapps/open_ended_grading/tests.py index b2529e38db162cc7ac733b9e203fd3da14c63e7d..556a4218081301130ae708bb88af518e0a3cb50f 100644 --- a/lms/djangoapps/open_ended_grading/tests.py +++ b/lms/djangoapps/open_ended_grading/tests.py @@ -1,7 +1,7 @@ """ Tests for open ended grading interfaces -django-admin.py test --settings=lms.envs.test --pythonpath=. lms/djangoapps/open_ended_grading +./manage.py lms --settings test test lms/djangoapps/open_ended_grading """ import json diff --git a/lms/envs/README.txt b/lms/envs/README.txt deleted file mode 100644 index 6d1512e6f6f80b1936201d1776297b3b9fb15bdd..0000000000000000000000000000000000000000 --- a/lms/envs/README.txt +++ /dev/null @@ -1,16 +0,0 @@ -Transitional for moving to new settings scheme. - -To use: - rake lms - or - django-admin.py runserver --settings=lms.envs.dev --pythonpath=. - -NOTE: Using manage.py will automatically run mitx/settings.py first, regardless -of what you send it for an explicit --settings flag. It still works, but might -have odd side effects. Using django-admin.py avoids that problem. -django-admin.py is installed by default when you install Django. - -To use with gunicorn_django in debug mode: - - gunicorn_django lms/envs/dev.py - diff --git a/manage.py b/manage.py new file mode 100755 index 0000000000000000000000000000000000000000..cde8f27e38165a5b4e8d2eee7e3dc23f6d793273 --- /dev/null +++ b/manage.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python +""" +Usage: manage.py {lms|cms} [--settings env] ... + +Run django management commands. Because edx-platform contains multiple django projects, +the first argument specifies which project to run (cms [Studio] or lms [Learning Management System]). + +By default, those systems run in with a settings file appropriate for development. However, +by passing the --settings flag, you can specify what environment specific settings file to use. + +Any arguments not understood by this manage.py will be passed to django-admin.py +""" + +import os +import sys +import glob2 +import imp +from argparse import ArgumentParser + +def parse_args(): + """Parse edx specific arguments to manage.py""" + parser = ArgumentParser() + subparsers = parser.add_subparsers(title='system', description='edx service to run') + + lms = subparsers.add_parser( + 'lms', + help='Learning Management System', + add_help=False, + usage='%(prog)s [options] ...' + ) + lms.add_argument('-h', '--help', action='store_true', help='show this help message and exit') + lms.add_argument( + '--settings', + help="Which django settings module to use. Will search for a submodule of lms.envs first, and " + "if that module doesn't exist, will search in sys.path. If not provided, the DJANGO_SETTINGS_MODULE " + "environment variable will be used if it is set, otherwise will default to lms.envs.dev") + lms.add_argument( + '-s', '--service-variant', + choices=['lms', 'lms-xml', 'lms-preview'], + default='lms', + help='Which service variant to run, when using the aws environment') + lms.set_defaults( + help_string=lms.format_help(), + settings_base='lms/envs', + default_settings='lms.envs.dev' + ) + + cms = subparsers.add_parser( + 'cms', + help='Studio', + add_help=False, + usage='%(prog)s [options] ...' + ) + cms.add_argument( + '--settings', + help="Which django settings module to use. Will search for a submodule of cms.envs first, and " + "if that module doesn't exist, will search in sys.path. If not provided, the DJANGO_SETTINGS_MODULE " + "environment variable will be used if it is set, otherwise will default to cms.envs.dev") + cms.add_argument('-h', '--help', action='store_true', help='show this help message and exit') + cms.set_defaults( + help_string=cms.format_help(), + settings_base='cms/envs', + default_settings='cms.envs.dev', + service_variant='cms' + ) + + + edx_args, django_args = parser.parse_known_args() + + if edx_args.help: + print "edX:" + print edx_args.help_string + + return edx_args, django_args + + +if __name__ == "__main__": + edx_args, django_args = parse_args() + + if edx_args.settings: + try: + imp.find_module(edx_args.settings, [edx_args.settings_base]) + os.environ["DJANGO_SETTINGS_MODULE"] = edx_args.settings_base.replace('/', '.') + "." + edx_args.settings + except ImportError: + imp.find_module(edx_args.settings) + os.environ["DJANGO_SETTINGS_MODULE"] = edx_args.settings + else: + os.environ.setdefault("DJANGO_SETTINGS_MODULE", edx_args.default_settings) + os.environ.setdefault("SERVICE_VARIANT", edx_args.service_variant) + if edx_args.help: + print "Django:" + # This will trigger django-admin.py to print out its help + django_args.insert(0, '--help') + + from django.core.management import execute_from_command_line + + execute_from_command_line([sys.argv[0]] + django_args) \ No newline at end of file diff --git a/rakelib/django.rake b/rakelib/django.rake index 53c299ac9f34e3aed902b7935210cd716ad467d6..40005e19e3cc41e41a92a516e2cd24495cb78f34 100644 --- a/rakelib/django.rake +++ b/rakelib/django.rake @@ -11,8 +11,7 @@ end task :fastlms do # this is >2 times faster that rake [lms], and does not need web, good for local dev - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') - sh("#{django_admin} runserver --traceback --settings=lms.envs.dev --pythonpath=.") + sh("./manage.py lms runserver --traceback") end # Start :system locally with the specified :env and :options. @@ -36,9 +35,8 @@ end desc "Start #{system} Celery worker" task "#{system}_worker", [:options] => [:predjango] do |t, args| args.with_defaults(:options => default_options[system]) - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') command = 'celery worker' - sh("#{django_admin} #{command} --loglevel=INFO --settings=#{system}.envs.dev_with_worker --pythonpath=. #{args.join(' ')}") + sh("./manage.py #{system} --settings dev_with_worker #{command} --loglevel=INFO #{args.join(' ')}") end # Per environment tasks diff --git a/rakelib/helpers.rb b/rakelib/helpers.rb index 925b7e1b2856551a56296381b840b95c921af6a5..6826570a1800108d8cff46de1ca9f897aaa50392 100644 --- a/rakelib/helpers.rb +++ b/rakelib/helpers.rb @@ -12,8 +12,7 @@ def select_executable(*cmds) end def django_admin(system, env, command, *args) - django_admin = ENV['DJANGO_ADMIN_PATH'] || select_executable('django-admin.py', 'django-admin') - return "#{django_admin} #{command} --traceback --settings=#{system}.envs.#{env} --pythonpath=. #{args.join(' ')}" + return "./manage.py #{system} --settings #{env} #{command} --traceback #{args.join(' ')}" end def report_dir_path(dir) diff --git a/scripts/create-dev-env.sh b/scripts/create-dev-env.sh index 8699cc76562b49fed9c0de760839ee3400ef566c..25e1b7520b4da9d9fa4b32ad27d2b1332ac15d91 100755 --- a/scripts/create-dev-env.sh +++ b/scripts/create-dev-env.sh @@ -495,8 +495,8 @@ mkdir -p "$BASE/log" mkdir -p "$BASE/db" mkdir -p "$BASE/data" -rake django-admin[syncdb,lms,dev,--noinput] -rake django-admin[migrate] +./manage.py lms syncdb --noinput --migrate +./manage.py cms syncdb --noinput --migrate # Configure Git @@ -520,18 +520,13 @@ if [[ ! $quiet ]]; then $ workon mitx - To initialize Django - - $ rake django-admin[syncdb] - $ rake django-admin[migrate] - To start the Django on port 8000 $ rake lms Or to start Django on a different <port#> - $ rake django-admin[runserver,lms,dev,<port#>] + $ ./manage.py lms runserver <port#> If the Django development server starts properly you should see: diff --git a/scripts/runone.py b/scripts/runone.py index a644aa077bd07e4f11150d378e85411634b31ef5..b403b09ff9574f070bf07df2934dc46ad01e6fa5 100755 --- a/scripts/runone.py +++ b/scripts/runone.py @@ -41,18 +41,17 @@ def main(argv): test_py_path = find_full_path(test_py_path) test_spec = "%s:%s.%s" % (test_py_path, test_class, test_method) - settings = None + system = None if test_py_path.startswith('cms'): - settings = 'cms.envs.test' + system = 'cms' elif test_py_path.startswith('lms'): - settings = 'lms.envs.test' + system = 'lms' - if settings: + if system: # Run as a django test suite from django.core import management - django_args = ["django-admin.py", "test", "--pythonpath=."] - django_args.append("--settings=%s" % settings) + django_args = ["./manage.py", system, "--settings", "test", "test"] if args.nocapture: django_args.append("-s") django_args.append(test_spec)