Commit 8b3c3b0d authored by Gerrit Hübbers's avatar Gerrit Hübbers 🃏
Browse files

Expand documentation on how to set up DDA in a production environment.

parent 66d9cce4
......@@ -7,66 +7,173 @@ To accomplish this aim, DDA consists of two complementary services:
## DDA and DSpace
Currently, one DDA installation supports one target DSpace 5+ repository installation. As a DDA installation is interacting with its target repository via REST, both can be deployed and restarted independently.
When a content provider has successfully used DDA to import a batch of content to the target DSpace repository, it will land in that collection's *XMLWorkflow* task pool, where that collection's editors and reviewers will have the chance to do their usual business of validating and improving the each submission before archiving (or rejecting) it.
Once a content provider has successfully used DDA to import a batch of content to the target DSpace repository, this content will land in the importing collection's *XMLWorkflow* task pool, where that collection's editors and reviewers will have the chance to do their usual business of validating and improving each submission before archiving (or rejecting) it.
# Initial setup
DDA is currently focused on working with a DSpace 5+ installation. In particular, it requires a running [DSpace REST endpoint](https://wiki.duraspace.org/display/DSDOC5x/REST+API) with [additional endpoints](https://git.gesis.org/dspace/rest-additions). DSpace must be running with [XMLWorkflow](https://wiki.duraspace.org/display/DSDOC5x/Configurable+Workflow).
## Initial setup
DDA is currently focused on working with a DSpace 5+ installation. In particular, it requires a running [DSpace REST endpoint](https://wiki.duraspace.org/display/DSDOC5x/REST+API) with [additional endpoints](https://git.gesis.org/dspace/rest-additions). DSpace must be running with [XMLWorkflow](https://wiki.duraspace.org/display/DSDOC5x/Configurable+Workflow) (not the XML-less *Workflow*).
## Creating a *Document Deposit Assistant* user
DDA will import documents to DSpace as a registered DSpace user. To create a new DDA user account within DSpace, first log in with administrator privileges. Then select *Access Control* -> *People*. Click *Click here to add a new E-Person*. Provide a valid and unique e-mail address, provide as first name "Document", as last name "Deposit Assistant", and have "Can Log In" selected. Click *Create E-Person*. Back in the *E-person management* interface, search for e-people with a string "Deposit Assistant", select the correct *Document Deposit Assistant* e-person from the results, and click *Login as E-Person* (in case it's available) or *Reset Password* in order to provide this user a password.
### Creating a *Document Deposit Assistant* DSpace user
DDA will import documents to DSpace as a registered DSpace user. To create a new DDA user account within DSpace, first log in with administrator privileges. Then select *Access Control* -> *People*. Click *Click here to add a new E-Person*. Provide a valid and unique e-mail address, provide as first name "Document", as last name "Deposit Assistant", and have *Can Log In* selected. Click *Create E-Person*. Back in the *E-person management* interface, search for e-people with a string "Deposit Assistant", select the correct *Document Deposit Assistant* e-person from the results, and click *Login as E-Person* (in case it is available) or *Reset Password* in order to provide this user a password.
## Creating a *Document Deposit Assistant* collection
### Creating a *Document Deposit Assistant* DSpace collection
DDA needs to know about a DSpace *collection* to which it can import its processed new items to.
In your DSpace installation, we suggest to create a new DSpace collection exclusively for DDA imports. This allows you to wipe all DDA-supplied imports in case something went wrong. While being logged in as a DSpace administrator, click on *Browse* -> *Communities & Collections* in order to get the *community list* overview. Either create a new community or select a community which you want the *Document Deposit Assistant* collection be part of, and click *create Collection*. Provide a meaningful name such as *Document Deposit Assistant* and click *Create*.
In your DSpace installation, we suggest to create a new DSpace collection exclusively for DDA imports. This allows you to easily wipe DDA-supplied imports in case something went wrong.
While being logged in as a DSpace administrator, click on *Browse* -> *Communities & Collections* in order to get the *community list* overview. Either create a new community or select a community which you want the *Document Deposit Assistant* collection to be part of, and click *create Collection*. Provide a meaningful name such as *Document Deposit Assistant* and click *Create*.
You will get into the *Edit Collection* dialog. On the *Assign Roles* tab, within the *submitters* section, click *Create...*. This will create a new group which is granted submitter rights to this collection; and you will be brought to the membership dialog for this group. Within this dialog, have a look at the headline. It should be of the form: `Group Editor: COLLECTION_XXX_SUBMIT (id: YYY)`. Keep note of the `XXX` part, as this is the collection *ID* (not collection *handle*) that we will require later. On this submitter group membership dialog, search for e-people with a string "Deposit Assistant", identify the correct *Document Deposit Assistant* e-person from the results, click on its *Add* button, and click *Save* to finalize this step.
### *Document Deposit Assistant* service daemon installation
DDA is a Java-based webservice. It also serves out a web browser user interface (using HTML and AngularJS).
Besides starting DDA directly from the command line (by running `mvn` from DDA's source code root directory), it is possible to setup DDA as a long running service daemon that persists transactional application data (user accounts, settings, in-transit publication metadata, etc.) to a database - in short, follow these steps to set up your own "production-ready" DDA instance.
In the following subsections, we will install DDA on an Ubuntu Linux machine. We will first setup the MySQL database, then prepare the filesystem, and finally install DDA as a Unix service which will automatically start up and shut down your DDA instance during Linux boot-up and shutdown, respectively.
#Technical overview
#### Database setup
DDA uses MySQL for persisting data. Assuming a MySQL server is running, and the `mysql` client tool exists, run the following commands in order to set up the DDA MySQL database in a state as expected by DDA's `staging` profile:
This application was generated using JHipster, you can find documentation and help at [https://jhipster.github.io](https://jhipster.github.io).
```
# The following command creates a new database called "dda" within your MySQL database engine.
# When prompted with *Enter password:* the mysql command expects you to provide the password of MySQL user *root*.
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="create database if not exists dda character set utf8 collate utf8_general_ci;"
# The following command creates a new user called "dda". Replace ${MYSQL_USER_DDA_PASSWORD} with a secret password and remember it.
# When prompted with *Enter password:* the mysql command expects you to provide the password of MySQL user *root*.
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="create user 'dda'@'localhost' identified by '${MYSQL_USER_DDA_PASSWORD}';"
# The following command grants the new user "dda" all database privileges on the "dda" database.
# When prompted with *Enter password:* the mysql command expects you to provide the password of MySQL user *root*.
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="grant all privileges on dda.* to 'dda'@'localhost'; flush privileges;"
```
#### Linux user and filesystem setup
Create a `dda` Linux user:
```
adduser --system --no-create-home --disabled-login --group dda
# Created this way, the `dda` user will have no login shell in order to increase security
# You will still be able to gain a shell for this user, by running the following command
# sudo -u dda bash
```
Update file `src/main/config/application-prod.yml` to reflect your production environment. In particular, provide correct values for the following keys:
* `spring.datasource.password`: this is the password of MySQL user `dda`.
* `spring.mail.(host, port, username, password`: these are settings for an SMTP mail server. Events such as user registration will send out e-mails using these settings.
* `server.port`: a TCP port that this DDA instance will listen on. Use a port which is not in use yet on your machine (e.g. 8081, if so).
* `ingester.endpoint`: your DSpace installation's REST endpoint, e.g. `https://www.example.com/rest`
* `ingester.email`: the e-mail address of the DSpace DDA user which you created in step [*Creating a Document Deposit Assistant DSpace user*](#creating-a-document-deposit-assistant-dspace-user)
* `ingester.password`: the password of aforementioned DSpace user
* `ingester.targetCollection`: the DSpace collection *ID* (not collection *handle*) of DDA's import collection which you created in step [*Creating a Document Deposit Assistant DSpace collection*](#creating-a-document-deposit-assistant-dspace-collection).
Now build a DDA production release. Run the following command:
```
mvn clean package -Pprod -DskipTests
```
Before you can build this project, you must install and configure the following dependencies on your machine:
This will create the executable web application artifact located at `target/dda-wizard.war`.
Next, create a directory where the DDA binary will reside:
```
sudo sh -c "mkdir /srv/dda && chown dda:dda /srv/dda && chmod u=rx,g=,o= /srv/dda"
# This command will create a directory `/srv/dda/`,
# set Linux user and group `dda` as its owner,
# and set minimal access rights to further increase security.
```
1. [Node.js][]: We use Node to run a development web server and build the project.
Depending on your system, you can install Node either from source or as a pre-packaged bundle.
Copy
* `target/dda-wizard.war` to `/srv/dda/dda-wizard.war` and
* `etc/conf-files/prod/dda-wizard.conf` to `/srv/dda/dda-wizard.conf`.
Set correct directory and file permissions:
```
sudo sh -c "chown -R dda:dda /srv/dda/ && chmod -R u=rx,g=,o= /srv/dda"
# This will set ownership of directory /srv/dda and all its content to user and group `dda`
# and set minimally required access rights in order to further increase security.
```
#### Service registration
Assuming you want to install DDA as a *SystemV init.d* service, first create a symlink from `/etc/init.d/dda-wizard` to the executable WAR file:
```
sudo ln -s /srv/dda/dda-wizard.war /etc/init.d/dda-wizard
```
Second, register DDA to start up and shut down during appropriate Linux lifecycle phases:
```
sudo update-rc.d dda-wizard defaults
# if you later want to deactivate DDA from starting automatically, run
# sudo update-rc.d -f dda-wizard remove
```
Now start up DDA:
```
sudo service dda-wizard start
# to shut down DDA, run `sudo service dda-wizard stop`
```
Start following the DDA log:
```
sudo tail -F /var/log/dda-wizard.log
```
The logs will let you know about the local and external IP addresses and ports on which DDA is listening, e.g. `http://127.0.0.1:8081` and `http://192.0.2.1:8081`, respectively.
* Try curling the local address: `curl http://127.0.0.1:8081`. You should get DDA's landing page HTML content returned on `stdout`.
* Try visiting the external address with your browser.
#### Reverse proxy configuration
TODO Apache configuration, tracked in issue #32.
#### System user passwords
TODO how to change passwords, tracked in issue #33.
# Technical overview
This application was generated using [JHipster 2.27.1](https://jhipster.github.io/v2-documentation/).
Before you can build this project, you must go through the following steps once.
DDA uses [Node.js](https://nodejs.org/) to build the project and run a development web server. Depending on your system, you can install Node.js either from source or as a pre-packaged bundle.
DDA was generated with jHipster 2.27.1. If you want to extend the DDA source code base with additional jHipster entities, you will need exactly that version:
npm install -g yo
npm install -g generator-jhipster@2.27.1
We use [Grunt][] as our build system. Install the grunt command-line tool globally with:
[Grunt](http://gruntjs.com/) is DDA's front-end build system. Install the grunt command-line tool globally with:
npm install -g grunt-cli bower
After installing Node, you should be able to run the following command to install development tools (like
[Bower][] and [BrowserSync][]). You will only need to run this command when dependencies change in package.json.
After installing Node, run the following command to install front-end development tools such as [Bower](http://bower.io/) and [BrowserSync](http://www.browsersync.io/). You will only need to run this command whenever front-end dependencies change in file `package.json`:
npm install
Run the following commands in two separate terminals to create a blissful development experience where your browser
auto-refreshes when files change on your hard drive.
Run the following commands in two separate terminals to create a blissful development experience where your browser auto-refreshes whenever DDA project files change in your filesystem.
mvn
grunt
mvn && grunt
Bower is used to manage CSS and JavaScript dependencies used in this application. You can upgrade dependencies by
specifying a newer version in `bower.json`. You can also run `bower update` and `bower install` to manage dependencies.
Bower is used to manage CSS and JavaScript dependencies. You can upgrade dependencies by specifying a newer version in file `bower.json`. You can also run `bower update` and `bower install` to manage dependencies.
Add the `-h` flag on any command to see how you can use it. For example, `bower update -h`.
## Staging
### Initial setup
The staging environment shall be as close as possible to the production environment. DDA uses MySQL in its staging environment. Assuming a MySQL server is running, and the `mysql` client tool exists, run the following commands in order to set up the DDA MySQL database in a state as expected by DDA's `staging` profile:
```
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="create database if not exists dda character set utf8 collate utf8_general_ci;"
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="create user 'dda'@'localhost' identified by 'dda';"
mysql --user=root --password --host=localhost --port=3306 --protocol=TCP --verbose --execute="grant all privileges on dda.* to 'dda'@'localhost'; flush privileges;"
```
The staging environment shall be as close as possible to the production environment. Therefore, run the same commands as listed in the [Database setup][#database-setup] chapter.
### Debugging staging environment
The staging environment is set up in such a way that it allows connecting a remote Java debugger (see file `etc/dda-wizard-staging.conf)`. You can connect to it like so:
The staging environment is set up in such a way that it allows connecting a remote Java debugger (see file `etc/conf-files/staging/dda-wizard.conf`). You can connect to it like so:
* First, ssh tunnel port-forward the remote debugger port with `ssh -L 8002:localhost:8002 svko-dda-test.gesis.intra`.
* Then, from Eclipse, create a new debug configuration with parameters `localhost` and port `8002`. Click connect.
......@@ -89,12 +196,11 @@ To ensure everything worked, run:
Then navigate to [http://localhost:8080](http://localhost:8080) in your browser.
## Testing
Unit tests are run by [Karma][] and written with [Jasmine][]. They're located in `src/test/javascript` and can be run with:
Unit tests are run by [Karma](http://karma-runner.github.io/) and written with [Jasmine](http://jasmine.github.io/2.0/introduction.html). They are located in `src/test/javascript` and can be run with:
grunt test
To test only the wizard, you can run:
To only test the wizard, you can run:
mvn test -Dtest=org.gesis.dda.wizard.**.*Test
......@@ -274,11 +380,3 @@ Let's now assume we are logged in, we are currently on the root `home` state, th
State `bundlesSource` is defined in file `bundlesSource.js`, with a `url=/bundlesSources`. The state hierarchy is `bundlesSource -> entity -> site`. Also this state fills the `content@` view with its own template `bundlesSources.html` and controller `BundlesSourceController`. That controller interacts with the `BundlesSource` $resource REST service (`bundlesSource.service.js`) - e.g., everytime this controller is activated, it will call `$scope.loadAll()`, concurrently populating the `$scope.bundlesSources` array with data returned from the remote REST endpoint.
[JHipster]: https://jhipster.github.io/
[Node.js]: https://nodejs.org/
[Bower]: http://bower.io/
[Grunt]: http://gruntjs.com/
[BrowserSync]: http://www.browsersync.io/
[Karma]: http://karma-runner.github.io/
[Jasmine]: http://jasmine.github.io/2.0/introduction.html
[Protractor]: https://angular.github.io/protractor/
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment