2018-10-03 Fred Gleason <fredg@paravelsystems.com>

* Updated 'UPGRADING' to include information for v3.x.
	* Updated 'INSTALL'.

ChangeLog

@@ -17740,3 +17740,6 @@
 	* Removed the POSTGRESQL_BUILTIN_DATABASE define from 'lib/rd.h'.
 2018-10-03 Fred Gleason <fredg@paravelsystems.com>
 	* Added a beta test warning to 'README'.
+2018-10-03 Fred Gleason <fredg@paravelsystems.com>
+	* Updated 'UPGRADING' to include information for v3.x.
+	* Updated 'INSTALL'.

INSTALL

@@ -8,53 +8,56 @@ Apache Web Server
 Included with most distros, or available from: http://www.apache.org/.
 
 Cdda2Wav
-A CD ripper engine.  Included in most distributions, but also available from 
+A CD ripper engine. Included in most distributions, but also available from 
 http://www.cdda2wav.de/.
 
 Expat
 A stream-oriented XML parser library. Available at https://libexpat.github.io/.
 
 ID3Lib
-An audio metadata tag library.  Included in most distros, or available at 
+An audio metadata tag library. Included in most distros, or available at 
 http://id3lib.sourceforge.net/.
 
 LibCurl, v7.19.0 or later
-A client-side URL transfer library.  Included with most distros, or
+A client-side URL transfer library. Included with most distros, or
 available at: http://curl.haxx.se/libcurl/.
 
 LibParanoia
-A library for ripping audio CDs.  Included in most distributions, but also 
+A library for ripping audio CDs. Included in most distributions, but also 
 available from http://www.xiph.org/paranoia/.
 
 LibSndFile
-An audio file support library, written by Erik de Castro Lopo.  Included with
+An audio file support library, written by Erik de Castro Lopo. Included with
 most distros, or you can find it at http://www.mega-nerd.com/libsndfile/.
 
-mySQL Database Server
+MySQL/MariaDB Database Server
-Included in most Linux distributions.  See http://www.mysql.com/.
+Included in most Linux distributions. See http://www.mysql.com/.
 
 PAM Pluggable Authentication Modules
 A suite of shared libraries that enable the local system administrator to 
-choose how applications authenticate users.  Included with virtually all modern
+choose how applications authenticate users. Included with virtually all modern
 distros, or see http://www.kernel.org/pub/linux/libs/pam/.
 
-OggVorbis - Open Source Audio Coding Library.  Needed for OggVorbis
+OggVorbis - Open Source Audio Coding Library. Needed for OggVorbis
-importing and exporting.  Included with most distros, or available at: 
+importing and exporting. Included with most distros, or available at: 
 http://www.xiph.org/.
 
 Qt Toolkit, v4.6 or better
-Most modern Linux distros include this.  It's typically installed as part
+Most modern Linux distros include this. It's typically installed as part
 of the KDE Desktop Environment, although KDE is by no means required.
 It can also be downloaded directly from TrollTech (http://www.trolltech.com/).
 
 Secret Rabbit Code
-A sample-rate converter library, written by Erik de Castro Lopo.  Included
+A sample-rate converter library, written by Erik de Castro Lopo. Included
 with most distros, or you can find it at http://www.mega-nerd.com/SRC/.
 
 SoundTouch Audio Processing Library
 A library for altering the pitch and/or tempo of digital audio data.
 Available at http://www.surina.net/soundtouch/.
 
+Systemd System and Service Manager
+Most modern Linux distros include this.
+
 X11 Window System
 Virtually all Linux distros should include this.
 
@@ -63,15 +66,15 @@ OPTIONAL PREREQUISITES
 The following components are optional, but needed at build- and run- time in
 order for particular features to work:
 
-One or more audio driver libraries.  Choices are:
+One or more audio driver libraries. Choices are:
 
   AudioScience HPI Driver - v3.00 or greater.
-  For supporting AudioScience's line of high-end professional audio adapters.  
+  For supporting AudioScience's line of high-end professional audio adapters.
   See http://www.audioscience.com/.
 
   The JACK Audio Connection Kit
   A low latency audio server, designed from the ground up for
-  professional audio work.  See http://jackit.sourceforge.net/.
+  professional audio work. See http://jackit.sourceforge.net/.
   Further information on running Rivendell with the JACK driver can be
   found in 'docs/JACK.txt'.
 
@@ -80,45 +83,41 @@ One or more audio driver libraries.  Choices are:
   See http://www.alsa-project.org/.
 
 Free Lossless Audio Codec (FLAC), v1.2.x or greater
-A "lossless" audio encoding library.  Included with most distros, or 
+A "lossless" audio encoding library. Included with most distros, or 
 available from: http://flac.sourceforge.net/.
 
-GPIO Driver
+FAAD2 / mp4v2 - AAC/MP4 Decoding Libraries. Needed for MP4 file importation.
-A kernel driver for the line of data-acquisition boards from 
-MeasurementComputing.  See http://www.rivendellaudio.org/.
-
-FAAD2 / mp4v2 - AAC/MP4 Decoding Libraries.  Needed for MP4 file importation.
 Available at http://www.audiocoding.com/faad2.html and
 https://code.google.com/p/mp4v2/ respectively.
 
-LAME - MPEG Layer 3 Encoder Library.  Needed for MPEG Layer 3 exporting.
+LAME - MPEG Layer 3 Encoder Library. Needed for MPEG Layer 3 exporting.
 Available at http://lame.sourceforge.net/.
 
-MAD - MPEG Audio Decoder Library.  Needed for MPEG importing and playout.
+MAD - MPEG Audio Decoder Library. Needed for MPEG importing and playout.
 Available at http://www.underbit.com/products/mad/.
 
-TwoLAME - MPEG Layer 2 Encoder Library.  Needed for MPEG Layer 2 exporting and
+TwoLAME - MPEG Layer 2 Encoder Library. Needed for MPEG Layer 2 exporting and
-capture.  Available at http://www.twolame.org/.
+capture. Available at http://www.twolame.org/.
 
 
 DOCUMENTATION
 -------------
 The larger pieces of the Rivendell documentation are written in XML-DocBook5.
 Pre-generated docs are included in the source tarball, so special tools will
-not normally be required to view or install them.  However, if you need to
+not normally be required to view or install them. However, if you need to
 rebuild them (either because you've modified the DocBook sources or are
 installing from the primary GitHub repository), then you will need the
 following:
 
-XML-DocBook5 Stylesheets.  Available at 
+XML-DocBook5 Stylesheets. Available at 
-http://sourceforge.net/projects/docbook/.  You will also need to create a
+http://sourceforge.net/projects/docbook/. You will also need to create a
 $DOCBOOK_STYLESHEETS variable in your environment that points to the top
-of the stylesheet tree.  More information can be found at
+of the stylesheet tree. More information can be found at
-http://www.docbook.org/tdg5/en/html/appa.html#s.stylesheetinstall.  On
+http://www.docbook.org/tdg5/en/html/appa.html#s.stylesheetinstall. On
 RHEL-ish systems, they are also available in the 'docbook5-style-xsl'
 package.
 
-xsltproc.  Command line XSLT processor.  Available at
+xsltproc. Command line XSLT processor. Available at
 http://xmlsoft.org/XSLT/xsltproc2.html
 
 Apache FOP. Formatting Objects (FO) processor. Available at
@@ -126,7 +125,7 @@ https://xmlgraphics.apache.org/fop/.
 
 INSTALLATION
 There are three major steps to getting a Rivendell system up and
-running.  They are:
+running. They are:
 
 1)  Setting up pre-requisite software
 
@@ -138,16 +137,16 @@ running.  They are:
 1)  Setting Up Prerequisites
 
 The major prerequisite piece of software needed for a functioning
-Rivendell system is the mySQL database engine.  This needs to
+Rivendell system is the mySQL database engine. This needs to
 be accessible from the target system (either by running on the local
 host, or on a remote system) before Rivendell installation proper
-is commenced.  In practice, this means that the 'mysqld' daemon is
+is commenced. In practice, this means that the 'mysqld' daemon is
-running and can be connected to using the mysql(1) client.  You will
+running and can be connected to using the mysql(1) client. You will
 also need a login name/password for an account on the server with
 administrative rights.
 
 The process of configuring mySQL on a given host can be intricate and
-is generally beyond the scope of this document.  Details can be found
+is generally beyond the scope of this document. Details can be found
 in a number of books on the subject, as well as in the very extensive
 documentation that accompanies the server itself.
 
@@ -160,11 +159,11 @@ list of currently supported GPIO boards are included in the package.
 
 Once the prerequisites are set up, installation is most often a matter of 
 cd'ing to the top of the Rivendell source tree and typing
-'./configure', 'make', followed (as root) by 'make install'.  Those
+'./configure', 'make', followed (as root) by 'make install'. Those
 who obtained the source via CVS will need to do './autogen.sh' first.
 
 The ./configure script will auto-detect what sound drivers (HPI, JACK
-or ALSA) are available and enable build support accordingly.  To override
+or ALSA) are available and enable build support accordingly. To override
 this behavior, it's possible to specify '--disable-hpi',
 '--disable-jack' or '--disable-alsa' as an argument to './configure'.
 Be sure to see the important additional information regarding
@@ -174,12 +173,12 @@ plan on using those sound driver architectures.
 The installation of Rivendell's web services components are controlled
 by two parameters passed to 'configure', as follows:
 
---libexecdir	  Location to install web scripts and static content
+--libexecdir	 Location to install web scripts and static content
 
---sysconfdir	  Location to install Apache configuration
+--sysconfdir	 Location to install Apache configuration
 
 The specific values to pass will vary widely depending upon the specific
-distro in question.  Some known good values (assuming a default Apache
+distro in question. Some known good values (assuming a default Apache
 setup):
 
 RHEL 5:	  --libexecdir=/var/httpd/rd-bin --sysconfdir=/etc/httpd/conf.d
@@ -194,9 +193,9 @@ After doing 'make install', be sure to restart the Apache web service.
 3)  Initial Configuration
 
 Next, you'll need to install a small configuration file at
-'/etc/rd.conf'.  A sample can be found in 'conf/rd.conf-sample'.  Much
+'/etc/rd.conf'. A sample can be found in 'conf/rd.conf-sample'. Much
 of this can be used unchanged, with the exception of the entries in the 
-[Identity] section.  These should be changed to reflect the user and group 
+[Identity] section. These should be changed to reflect the user and group 
 name of the system accounts that will be running Rivendell.
 
 The directory for the audio sample data next needs to be created, as
@@ -208,16 +207,32 @@ This directory should owned, readable, writable and searchable by the user
 and group specified in the 'AudioOwner=' and 'AudioGroup=' entires in
 '/etc/rd.conf' and readable and searchable by Others (mode 0775).
 
-Finally, it's time to start things up.  Run 'rdadmin' from a shell
+Next, create an empty database on the MySQL/MariaDb server, as well as a
-prompt.  For the first time startup, RDAdmin will prompt for a login
+DB user to access it. This user should have the following privileges:
-name/password on the mySQL server so that it can create the Rivendell
-database.  To log into RDAdmin for the first time, enter a User Name of
-'admin' with no password.
 
-Much of the work in Rivendell gets done by three daemon processes,
+       Select
-named 'caed', 'ripcd' and 'rdcatchd'.  These daemons *must* be running
+       Insert
-before attempting to start any of the Rivendell applications.  The
+       Update
-order in which they are started is important, and should be the same
+       Delete
-as the order in which they are listed above.  For convienence, a Sys-V
+       Create
-style init script called 'rivendell' that can start, stop and restart 
+       Drop
-the daemons properly is installed in '/etc/init.d/'.
+       References
+       Index
+       Alter
+       Create Temporary Table
+       Lock Tables
+
+In the '[mySQL]' section of the '/etc/rd.conf' file, set the 'Database=',
+'Loginname=' and 'Password=' parameters to the DB name, user and password
+that you created. Then, create an initial Rivendell database by doing:
+
+     rddbmgr --create
+
+If all goes well, this command should return with no output.
+
+Finally, start up the Rivendell service by doing (as root):
+
+      systemctl start rivendell
+
+You should now be able to run the various Rivendell components from the
+Applications menu.

UPGRADING

@@ -1,25 +1,25 @@
 IMPORTANT NOTE: BE SURE TO SEE THE SECTION ON 'PASSWORD STORAGE' BELOW,
 OTHERWISE YOU MAY BE LOCKED OUT OF THE SYSTEM AFTER THE UPDATE!
 
-This file provides some notes to those performing an upgrade from a v1.x
+This file provides some notes to those performing an upgrade from a v1.x or
-version to Rivendell.  It is an attempt to provide some pointers about things
+v2.x version of Rivendell. It is an attempt to provide some pointers about
-which have changed, but is in no way a substitute for reading the 'INSTALL' 
+things which have changed, but is in no way a substitute for reading the
-file and other documentation!
+'INSTALL' file and other documentation!
 
 1) DEPENDENCIES
 The set of external packages required to build and run Rivendell has changed
-significantly from that required for a 1.x system.  See 'INSTALL' for a full
+significantly from that required for a 1.x or 2.x system. See 'INSTALL' for
-list.
+a full list.
 
 2) MPEG SUPPORT
-This version of Rivendell inaugurates optional support for MPEG Layer 2 
+This version of Rivendell includes optional support for MPEG Layer 2 
 encoding within the core Rivendell audio library for all classes of audio 
-device (JACK and ALSA as well as AudioScience HPI).  To make use of this 
+device (JACK and ALSA as well as AudioScience HPI). To make use of this 
 feature, it will be necessary to have the appropriate MPEG libraries 
-properly installed at both build- and run-time.  See 'INSTALL' for the 
+properly installed at both build- and run-time. See 'INSTALL' for the 
 specifics.
 
-3) FILE OWNERSHIP AND PERMISSIONS
+3) FILE OWNERSHIP AND PERMISSIONS (v1.x only)
 Recommended file ownership and permissions for the '/var/snd' directory and 
 its contents have changed.  These items should be owned by system user and 
 group accounts created specifically for Rivendell; no 'real' user on the 
@@ -32,25 +32,25 @@ should have read, write set for user and group and just read for others (0664).
 
 4) WEB SERVICES
 This version of Rivendell makes use of a web services protocol to accomplish
-many functions (audio import, export, ripping, etc).  These services require
+many functions (audio import, export, ripping, etc). These services require
 that a CGI-compliant web server be installed and active on the system.  
 Any server that complies with CGI-1.1 should work, although as of this writing
-only Apache 2.2 has been well tested.  A configuration file snippet for Apache
+only Apache 2.2 has been well tested. A configuration file snippet for Apache
 that will configure the target web services directory (set by the
 '--libexecdir=' switch passed to './configure') correctly is generated 
 automatically as part of the build process; it can be found in
 'conf/rd-bin.conf' after the build is complete.
 
-5) PASSWORD STORAGE
+5) PASSWORD STORAGE (v1.x only)
-The method of storing passwords in the database has changed in Rivendell 2.x,
+The method of storing passwords in the database has changed in Rivendell 2.x
-requiring that all non-null passwords be reset after performing the upgrade.
+and 3.x, requiring that all non-null passwords be reset after performing the
-This means that it is important that the account used to access RDAdmin be set
+upgrade. This means that it is important that the account used to access
-to use a null (blank) password *before* applying the update, otherwise you
+RDAdmin be set to use a null (blank) password *before* applying the update,
-will not be able to access RDAdmin afterwards!  Once the update has been
+otherwise you will not be able to access RDAdmin afterwards!  Once the update
-applied, passwords should be re-entered for all user accounts in RDAdmin
+has been applied, passwords should be re-entered for all user accounts in
-(including administrative ones) in the usual manner.
+RDAdmin (including administrative ones) in the usual manner.
 
-6) AUDIO SAMPLE RATE
+6) AUDIO SAMPLE RATE (v1.x only)
 The sample rate to be used for the Rivendell audio library is now a single
 system-wide setting (found in RDAdmin->SystemSettings); it is no longer
 possible to set sample rates individually per module/workstation.  When 
@@ -67,8 +67,51 @@ the target file, respectively.
 
 The 'smb:' protocol is no longer supported.
 
-8) RLM API CHANGES
+8) RLM API CHANGES (V1.x only)
 The parameters passed to the RLMPadDataSent() callback in RLMs have changed;
 private plug-ins written for RD 1.x will need to be modified to work with the
 new API.  (All plug-ins supplied with the Rivendell sources have already been
 modified appropriately.)
+
+9) DATABASE CHARACTER SET CONVERSION
+Databases generated using Rivendell v1.x or v2.x used the 'latin1' character
+set. Such databases must be converted to use the 'utf8mb4' character set
+before they can be properly utilized by Rivendell 3.x. To perform this
+conversion, proceed as follows:
+
+BEFORE UPDATING
+---------------
+A) Make a full and complete backup of the database, using either rdadmin(1)
+or the MySQL mysqldump(1) utility. Don't skip this!
+
+B) Use the rddbcheck(8) tool to verify the consistency of the database.
+Be sure to fix any problems found *before* upgrading the Rivendell binaries!
+
+AFTER UPDATING
+--------------
+C) Check the '[mySQL]' section of '/etc/rd.conf' and verify that the
+following parameters are set:
+
+[mySQL]
+Charset=utf8mb4
+Collation=utf8mb4_general_ci
+
+(There will likely be additional parameters present in the '[mySQL]'
+section; these can be left unchanged).
+
+D) Update the DB schema using the rddbmgr(8) command:
+
+   rddbmgr --modify
+
+If all goes well, the command should exit without printing anything.
+
+E) Convert the DB character set using the rddbmgr(8) command:
+
+   rddbmgr --check --yes
+
+You should see a list of DB tables generated as each one is converted. 
+
+At this point, you should be ready to start the Rivendell service, by doing:
+
+   systemctl restart rivendell
+