Difference between revisions of "Wikispooks:Site Backup"

From Wikispooks
Jump to navigation Jump to search
(upgrade the technical requirements etc.)
 
(121 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{HelpSection|image=SiteBackup.png}}
 
{{HelpSection|image=SiteBackup.png}}
<div style="font-size:86%; text-align:center; border:1px solid #aaaaaa; background-color:#f9f9f9; color:black; margin-left:1em; padding:0em; float:right; clear:right;">
+
{{IframeFolder|path=/backups/?C=N;O=D}}
{{#widget:Iframe
+
<big>'''You are encouraged to download the <u class="plainlinks">[[image:compressed.gif|link={{server}}/backups/wikispooks-latest.zip]][{{server}}/backups/wikispooks-latest.zip &nbsp;wikispooks-latest]</u> backup ({{BackupSize}}) in case the site goes offline for any reason.'''<big/><hr/>
|url=https://wikispooks.com/backups
+
Weekly backup snapshots for the last two months are listed on the right in the '''<font face="Serif">Backup files list</font>'''. The <u class="plainlinks">[[image:compressed.gif|link={{server}}/backups/wikispooks-latest.zip]][{{server}}/backups/wikispooks-latest.zip <code>&nbsp;wikispooks-latest.zip</code>]</u> file is a link to the most recent snapshot ({{t|{{BackupSize}}}}). As the site is [[Wikispooks:Copyrights|open licenced]] you are free to publish some or all of the content elsewhere, either as a mirror or as part of another website.
|width=570
 
|height=550
 
|border=1
 
}}</div>
 
'''Even if you are a casual reader and not a contributor to this project, you are encouraged to download and keep a backup copy of the [{{server}}/backups/ws-latest.zip site database] (~100&nbsp;MB) and it's associated [{{server}}/backups/files-latest.zip files and images] (~2.3&nbsp;GB) in case this site goes offline for any reason.''' The site is [[WikiSpooks:Copyrights| open licenced]] so you are free to copy this information and publish it elsewhere, either as an exact mirror or as part of another website. The more people around the world who keep backups, the better the chances this information will not be lost. <!-- Backups of the [{{server}}/backups/mediawiki-latest.zip full Mediawiki software suite] (~50&nbsp;Mb), as configured for the site, are also available.-->
 
  
==Content of the 'Backups' directory==
+
==Setting up and running a Copy Of The Site==
Be sure to check the dates in the '''<font face="Serif">Index of /backups</font>''' box on the right. You will probably want those of the form <code>*-latest.zip</code>, since these are the most up to date.
+
{| class="wikitable" style="margin:.4em; float:right"
* [[image:file.gif|20px|link={{server}}/backups/LSadditions.txt]] <code>LSadditions.txt</code> &nbsp;&nbsp;&nbsp; ''- Text to add to the end of <code>LocalSettings.php</code>
+
| style="max-width:375px; text-align:left; padding:.4em .4em .4em 1em; font-size:105%; background-color:#EEE" | [[File:Gnome-system-software-installer.svg|40px]] <span style="font-size:118%; margin-left:10px;">System requirements</span>
* [[image:file.gif|20px|link={{server}}/backups/Readme.txt]] <code>Readme.txt</code> &nbsp;&nbsp;&nbsp; ''- a text rendering of this page''
+
|-
* [[image:compressed.gif|20px|link={{server}}/backups/files-latest.zip]] <code>files-YYYY-MM-DD.zip</code> &nbsp;&nbsp;&nbsp; ''- a ZIP file of the site images and files''
+
| style="padding:0 .4em .24em; font-size:95%; max-width:375px" |
* [[image:compressed.gif|20px|link={{server}}/backups/ws-latest.zip]] <code>ws-YYYY-MM-DD.zip</code> &nbsp;&nbsp;&nbsp; ''- a ZIP file of the site database''
+
* '''Disk space''' > 16Gb
* [[image:unknown.gif|20px|link={{server}}/backups/ws.sql]] <code>ws.sql</code> &nbsp;&nbsp;&nbsp; ''- uncompressed site database''
+
* '''Web server''' such as [http://httpd.apache.org Apache] or [http://www.microsoft.com/iis IIS] or [http://nginx.org/en/ NginX]
 +
* '''[http://www.php.net PHP]''' version 5.6 or later, including:
 +
** [http://php.net/spl Standard PHP Library]
 +
** [http://php.net/pcre pcre] - Perl Compatible Regular Expressions
 +
** [http://www.php.net//manual/en/book.json.php JSON]
 +
** [http://php.net/manual/en/book.image.php GD] - Thumbnailing graphics
 +
** [http://php.net/manual/en/book.xml.php XML]
 +
** [http://php.net/manual/en/book.mbstring.php mbstring] - Multibyte strings
 +
* '''Database Server'''
 +
** [http://www.mysql.com MySQL] version 5.5 or later, or
 +
** [https://mariadb.org/MariaDB MariaDB] version 10.0 or later
 +
|}
 +
''The [{{server}}/backups/wikispooks-latest.zip backup file] contains everything you need to run a mirror of this site on a suitably configured web server; Separate installation of Mediawiki is NOT required.''
 +
 
 +
The web server currently used by Wikispooks is '''[http://httpd.apache.org/ Apache]''' running on [https://www.centos.org/about/ Centos 7] Linux. These instructions assume Apache, though [http://nginx.org/en/ NginX], [http://www.microsoft.com/web/platform/server.aspx IIS] and other web servers may work just as well. The instructions have been slightly modified to try to suit other Linux distos as well.
  
==Overview==
+
'''N.B.''' ''[http://www.wampserver.com/en/ WampServer] or [http://www.apachefriends.org/en/xampp.html‎ Xampp] provide all the Apache web-server software required (except for Xcache) to run the Wikispooks site locally on either a Linux or Windows machine.''. If you do not install either [http://xcache.lighttpd.net/ Xcache] or [http://pecl.php.net/package/APC APC] separately, then you must change <code>$wgMainCacheType = CACHE_ACCEL;</code> to <code>$wgMainCacheType = CACHE_DB;</code> in <Code>/w/LocalSettings.php</code>.
To host Wikispooks on your machine, you will need:
 
* Some knowledge of running a webserver
 
* Some knowledge of the Mediawiki software
 
* A site-backup set - The necessary backup files are listed above and '''[{{server}}/backups/ here]'''.
 
* A computer with at least 5 Gb of free disk space that runs a web server, including
 
** PHP v 5.3.3 +
 
** MySQL v 5.1.73 +
 
** Lua v 5.1.5 +
 
''N.B. Either [http://www.wampserver.com/en/ WampServer] or [http://www.apachefriends.org/en/xampp.html‎ Xampp] provides the web-server software you need to run the Wikispooks site locally.''
 
  
==Setting up and running a Copy Of The Site==
+
===Overview===
The backups enable you to import all the files, images and text-content of the site into 'Mediawiki'. Your first task is therefore to install Mediawiki and some mandatory Mediawiki extensions. A list of ''all'' installed site extensions is [[Special:Version|shown here]]. Many of them are mandatory for the site to work properly.
+
Set up your web server; create a blank MySQL database; download <u class="plainlinks">[[image:compressed.gif|link={{server}}/backups/wikispooks-latest.zip]][{{server}}/backups/wikispooks-latest.zip <code>wikispooks-latest.zip</code>]</u>  to your server root and unpack it; import file <code>''path-to-server-root''/backups/ws.sql</code> into your blank MySQL database; edit <code>''path-to-server-root''/w/LocalSettings.php</code> and enter your database name and user credentials; navigate to your web-server root from your web browser to browse your local copy of Wikispooks.
  
===Mandatory extensions===
+
===Web server directives===
A list of mandatory extensions in approximate order of importance for a functional and presentable site:
+
*[http://www.mediawiki.org/wiki/Manual:Short_URL Pretty urls] are required for full site functionality. Add the following to the bottom of your server or virtual server config file:<br/>
*Semantic MediaWiki  - '''NB''': The site will not display ANY pages without this extension.
+
:<code>Alias /wiki ''path-to-server-root''/w/index.php</code>.<br/>
*Semantic Forms
+
:''Apache'' config files are usually located at <code>/etc/httpd/</code>.
*Semantic Result Formats
+
*for correct display of the [[Project:Site Backup|Site backups list]] box. Add the following to your server config file:<br/>
*ParserFunctions
+
:<poem><Code><Directory "''path-to-server-root''/backups">
*Scribunto
+
::Options +Indexes
*Cite
+
::IndexIgnore  .. header.html
*Poem
+
::HeaderName header.html
*Widgets
+
::IndexOptions FancyIndexing SuppressDescription SuppressLastModified NameWidth=*
*Gadgets
+
</Directory></code></poem>
*CategoryTree
+
*'''Linux only''': you may have to tweek a few directory permission for the site to run smoothly. In particular the following require user (Usually <code>Apache</code>) write permissions:<br/>
*No title
+
:<code>''path-to-server-root''/w/cache/</code>
*MagicNoCache
+
:<code>''path-to-server-root''/w/images/</code>
*MiniMp3
 
*SubString and StrLen
 
'''Note''' A few mandatory extensions are not listed above because they are included with a standard installation of one or more of those that are mentioned.
 
  
===The 'LocalSettings.php' file===
+
===The MySQL database===
Successful installation of the Mediawiki software requires that you download an install-generated file called <code>"LocalSettimgs.php"</code>, and put it into the Mediawiki root directory - usually <code>/serverRoot/w/</code>. The file is editable with any plain-text editor. The end of the file will contain the following text:
+
The database must be created manually (see [http://www.mediawiki.org/wiki/Manual:MySQL MySQL manual]).<br/> This site uses database name "<code>wikispooks</code>" and user "<code>WSuser</code>", so using these is simplest. If you change them, make sure to [[#Database settings|adjust '''{{t|LocalSettings.php}}''' accordingly]].
  
<pre>
+
===The backup file===
# End of automatically generated settings.
+
Place your downloaded <code>latest-site.zip</code> file in the server root directory and and unzip it. This will create a directory structure containing all application and associated site files. It will also place the database content file <code>ws.sql</code> in the <code>''path-to-server-root''/backups/</code> directory.
# Add more configuration options below.
 
</pre>
 
  
The wikispooks site requires a number of custom configuration parameters, most notably to define new namespaces, extension statements, and permissions. The easiest way for you to include everything necessary is to copy/paste the contents of <code>"LSadditions.txt"</code> to the end of the auto-generated part of the file. However, note that  <code>"LSadditions.txt"</code> includes ALL the wikispooks installed extension statements so, if you don't install ALL of them too, you may get errors reported. These can be dealt with by commenting out the corresponding extension statements.
+
===Import the Wikispooks content into your database===
 +
Import <code>''path-to-server-root/backups/''ws.sql</code> into the new database.  
  
Both the Mediawiki and extensions software, together with comprehensive installation instructions are available '''[http://www.mediawiki.org/wiki/MediaWiki from here]'''
+
'''N.B.''' The file is too large for a default configuration of phpMyAdmin to handle. Try a MySQL command for this, e.g.
 +
::<code>$ mysql -u username --password=''password'' --host=''localhost'' data-base-name < ws.sql </code>
  
====Error reporting====
+
===LocalSettings.php===
If things don't go smoothly, adding the following lines to LocalSettings.php will give you some diagnostic output:
+
The file  at <code>''path-to-server-root''/w/LocalSettings.php</code> contains the local customisations of the default Mediawiki install. It needs editing as follows to match your install:
::<code>$wgShowExceptionDetails = true;</code>
 
::<code>$wgShowSQLErrors = 1;</code>
 
It's recommended to take these lines out once things are working OK.
 
  
==Getting the wikispooks site files, images and content into Mediawiki==
+
====Domain settings====
Once you have a working installation of Mediawiki, proceed as follows:
+
* Modify the below two lines to suit your install (For installation on your machine, set these to "http" and "localhost" respectively):
*Unzip <code>"sqldump-latest.zip"</code> in a convenient directory. This will will generate a file named <code>"ws.sql"</code>
+
<poem><code>$WSprotocol='https'; // For a local install, should be "http"
*Unzip <code>"files-latest.zip"</code> '''in the web server root directory'''. This will generate many directories and files in and under the web server root.
+
$WSdomain='wikispooks.com'; // For a local install, should be "localhost"</code></poem>
*Import the ws.sql database dump into the new database.
 
** '''N.B.''' The file is too large for a default configuration of phpMyAdmin to handle. Try a MySQL command for this, e.g.
 
::<code>$ mysql -u username -p password -h localhost data-base-name < ws.sql </code>
 
  
===Finishing Touches===
+
====Database settings====
The procedure as explained above is sufficient for the database and all the files. There will be a number of Apache config parameters that need setting in order for 'pretty URLs' to work properly, such as a <code>.htaccess</code> file. Full details can be found on the [http://www.mediawiki.org/wiki/MediaWiki Mediawiki web site]. 
+
* Enter your new database details in the below lines:
 +
<poem><code>$wgDBtype = "mysql";
 +
$wgDBserver = "localhost";
 +
$wgDBname = "wikispooks";
 +
$wgDBuser = "root";
 +
$wgDBpassword = "";</code></poem>
  
==Running A Live Site Mirror==
+
==Troubleshooting==
 +
If things don't go smoothly, the first thing you should do is go to the end of the file where you will see the following lines:
 +
::<code>#$wgShowExceptionDetails = true;</code>
 +
::<code>#$wgShowSQLErrors = 1;</code>
 +
Remove the <code>#</code> characters to get diagnostic output that may help you debug. It's recommended to replace them once things are working OK.
  
If you have interest and resources to set up a realtime site mirror on your server, you are encouraged to do so. We may also be able to provide you with some technical assistance if needed. Requests for help setting up realtime mirrors should be addressed to <code>admin[at]wikispooks.org</code> and encrypted with [[WikiSpooks:PGP Public Key|'''this key''']].
+
===<small>{{t|Warning: include(../../cgi-bin/WsDbInfo.php)}}...</small>===
 +
This occurs if you haven't [[#Database settings|fixed the database credentials]] as explained above.
  
==Notes==
+
===<small>{{t|CACHE_ACCEL requested but no suitable object cache is present. }}</small>===
'''The following items are ''not'' included in the backup:'''
+
This means that the ACCEL cache is not available on your webserver. The first thing to try is to find the following lines:
* 'ISGP archive' - the full ISGP archive (133MB) is available from https://wikispooks.com/ISGP/ISGP_site_backup_Jan_2013.zip
+
::<code>$wgMainCacheType = CACHE_ACCEL;</code>
* The 'Anon Uploads Form'
+
::<code>#$wgMainCacheType = CACHE_DB;</code>
* 'ACH sytem'
+
Try commenting the first of these out, and uncommenting the second. If this doesn't work, the easiest way to get it running is to comment them both out. This disables the cache, which will impact performance, but that problem can be tackled later if need arises.
* 'Piwik stats'
 
* Wikileaks 'Cablegate Mirror'
 
* Wikileaks 'Torrents'
 
  
===Known Issues===
+
===<small>{{t|Sorry! This site is experiencing technical difficulties. }}</small>===
* You may experience difficulty with file permissions on the images directory during the install process. In this case, you may need to change these permissions. For more details, see http://www.mediawiki.org/wiki/Manual:Security#Upload_security
+
The active part of this message is a couple of lines lower. If it says "Cannot contact the database server:" this means that [[#Insert correct Database credentials|the custom Database credentials]] you entered into Localsettings.php are not functioning properly.
  
* Many of the in-text links (especially on 'Category:IFrame' pages) will take you to the master Wikispooks site. This is because urls which ought to be coded as relative, have in fact been entered as absolute urls. It is easy to manually change any such url's in your browser address-bar at run time (simply replace 'https://wikispooks.com/' with 'localhost/your_root_dir_name/'. This issue will be addressed at source as and when.
+
===<small>{{t|Lua error: Internal error: The interpreter exited with status 2}}</small>===
 +
===<small>{{t|Lua error: Internal error: Error reading from pipe.}}</small>===
 +
The above error messages indicate that Lua is hitting a memory limit problem.<ref>https://www.mediawiki.org/wiki/Topic:Sftr334fvttqt3qm</ref> In this case, the pages will appear mostly OK, but will lack infoboxes. In this case, you should raise the amount of memory available for Lua.<ref>https://www.mediawiki.org/wiki/Extension:Scribunto#Lua_error:_Internal_error:_The_interpreter_exited_with_status_2</ref> Find the following two lines and comment them in:
 +
::<code>#$wgScribuntoEngineConf['luastandalone']['memoryLimit'] = 209715200; # bytes</code>
 +
::<code>#$wgMaxShellMemory = 204800; # in KB</code>
  
* '''Suggestions, Queries etc? - Either Email the admin (non-WikiSpooks members) or use [[WikiSpooks talk:Site Backup|this discussion page]]''' (WikiSpooks members)
+
==Running A Live Site Mirror==
 +
If you have interest and resources to set up a realtime site mirror on your server, you are encouraged to do so. We may also be able to provide you with some technical assistance if needed. Requests for help setting up realtime mirrors should be addressed to <code>admin[at]wikispooks.org</code> and encrypted with [[Wikispooks:PGP Public Key|'''this key''']].
  
[[Category:WikiSpooks Technical]]
+
===Known Issues===
 +
* [http://xdebug.org/ xdebug] is a php debugging tool which produces output similar (but considerably more verbose) to <code>$wgShowExceptionDetails</code> above. It is installed and enabled by default in both ''Wamp server'' and ''Xampp''. Wikispooks admins have found it useful in tracking down common installation problems but that a default setting results in ungraceful ''out of memory'' failures on some of the more common installation issues, rendering it useless in tracking them down. Either disable it by commenting out the ''[xdebug]'' block in ''php.ini'' or increase <code>max_nesting_level</code> from the default 100 with the following if you wish to use it: <code>xdebug.max_nesting_level=300</code>
 +
* The backups do not include the [{{server}}/anon/anon-ul.html Anon Uploads Form] or [{{server}}/piwik/index.php Piwik stats].
 +
* '''Suggestions, Queries etc? - Either Email the admin (non-Wikispooks members) or use [[Wikispooks talk:Site Backup|this discussion page]]''' (Wikispooks members)
 +
{{Wikispooks Footer/Technical}}
 +
[[Category:Wikispooks Technical]]

Latest revision as of 19:36, 24 May 2021

SiteBackup.png

You are encouraged to download the Compressed.gif wikispooks-latest backup (13 Gb) in case the site goes offline for any reason.


Weekly backup snapshots for the last two months are listed on the right in the Backup files list. The Compressed.gif wikispooks-latest.zip file is a link to the most recent snapshot (13 Gb). As the site is open licenced you are free to publish some or all of the content elsewhere, either as a mirror or as part of another website.

Setting up and running a Copy Of The Site

Gnome-system-software-installer.svg System requirements

The backup file contains everything you need to run a mirror of this site on a suitably configured web server; Separate installation of Mediawiki is NOT required.

The web server currently used by Wikispooks is Apache running on Centos 7 Linux. These instructions assume Apache, though NginX, IIS and other web servers may work just as well. The instructions have been slightly modified to try to suit other Linux distos as well.

N.B. WampServer or Xampp provide all the Apache web-server software required (except for Xcache) to run the Wikispooks site locally on either a Linux or Windows machine.. If you do not install either Xcache or APC separately, then you must change $wgMainCacheType = CACHE_ACCEL; to $wgMainCacheType = CACHE_DB; in /w/LocalSettings.php.

Overview

Set up your web server; create a blank MySQL database; download Compressed.gifwikispooks-latest.zip to your server root and unpack it; import file path-to-server-root/backups/ws.sql into your blank MySQL database; edit path-to-server-root/w/LocalSettings.php and enter your database name and user credentials; navigate to your web-server root from your web browser to browse your local copy of Wikispooks.

Web server directives

  • Pretty urls are required for full site functionality. Add the following to the bottom of your server or virtual server config file:
Alias /wiki path-to-server-root/w/index.php.
Apache config files are usually located at /etc/httpd/.
  • for correct display of the Site backups list box. Add the following to your server config file:

<Directory "path-to-server-root/backups">
Options +Indexes
IndexIgnore .. header.html
HeaderName header.html
IndexOptions FancyIndexing SuppressDescription SuppressLastModified NameWidth=*
</Directory>

  • Linux only: you may have to tweek a few directory permission for the site to run smoothly. In particular the following require user (Usually Apache) write permissions:
path-to-server-root/w/cache/
path-to-server-root/w/images/

The MySQL database

The database must be created manually (see MySQL manual).
This site uses database name "wikispooks" and user "WSuser", so using these is simplest. If you change them, make sure to adjust LocalSettings.php accordingly.

The backup file

Place your downloaded latest-site.zip file in the server root directory and and unzip it. This will create a directory structure containing all application and associated site files. It will also place the database content file ws.sql in the path-to-server-root/backups/ directory.

Import the Wikispooks content into your database

Import path-to-server-root/backups/ws.sql into the new database.

N.B. The file is too large for a default configuration of phpMyAdmin to handle. Try a MySQL command for this, e.g.

$ mysql -u username --password=password --host=localhost data-base-name < ws.sql

LocalSettings.php

The file at path-to-server-root/w/LocalSettings.php contains the local customisations of the default Mediawiki install. It needs editing as follows to match your install:

Domain settings

  • Modify the below two lines to suit your install (For installation on your machine, set these to "http" and "localhost" respectively):

$WSprotocol='https'; // For a local install, should be "http"
$WSdomain='wikispooks.com'; // For a local install, should be "localhost"

Database settings

  • Enter your new database details in the below lines:

$wgDBtype = "mysql";
$wgDBserver = "localhost";
$wgDBname = "wikispooks";
$wgDBuser = "root";
$wgDBpassword = "";

Troubleshooting

If things don't go smoothly, the first thing you should do is go to the end of the file where you will see the following lines:

#$wgShowExceptionDetails = true;
#$wgShowSQLErrors = 1;

Remove the # characters to get diagnostic output that may help you debug. It's recommended to replace them once things are working OK.

Warning: include(../../cgi-bin/WsDbInfo.php)...

This occurs if you haven't fixed the database credentials as explained above.

CACHE_ACCEL requested but no suitable object cache is present.

This means that the ACCEL cache is not available on your webserver. The first thing to try is to find the following lines:

$wgMainCacheType = CACHE_ACCEL;
#$wgMainCacheType = CACHE_DB;

Try commenting the first of these out, and uncommenting the second. If this doesn't work, the easiest way to get it running is to comment them both out. This disables the cache, which will impact performance, but that problem can be tackled later if need arises.

Sorry! This site is experiencing technical difficulties.

The active part of this message is a couple of lines lower. If it says "Cannot contact the database server:" this means that the custom Database credentials you entered into Localsettings.php are not functioning properly.

Lua error: Internal error: The interpreter exited with status 2

Lua error: Internal error: Error reading from pipe.

The above error messages indicate that Lua is hitting a memory limit problem.[1] In this case, the pages will appear mostly OK, but will lack infoboxes. In this case, you should raise the amount of memory available for Lua.[2] Find the following two lines and comment them in:

#$wgScribuntoEngineConf['luastandalone']['memoryLimit'] = 209715200; # bytes
#$wgMaxShellMemory = 204800; # in KB

Running A Live Site Mirror

If you have interest and resources to set up a realtime site mirror on your server, you are encouraged to do so. We may also be able to provide you with some technical assistance if needed. Requests for help setting up realtime mirrors should be addressed to admin[at]wikispooks.org and encrypted with this key.

Known Issues

  • xdebug is a php debugging tool which produces output similar (but considerably more verbose) to $wgShowExceptionDetails above. It is installed and enabled by default in both Wamp server and Xampp. Wikispooks admins have found it useful in tracking down common installation problems but that a default setting results in ungraceful out of memory failures on some of the more common installation issues, rendering it useless in tracking them down. Either disable it by commenting out the [xdebug] block in php.ini or increase max_nesting_level from the default 100 with the following if you wish to use it: xdebug.max_nesting_level=300
  • The backups do not include the Anon Uploads Form or Piwik stats.
  • Suggestions, Queries etc? - Either Email the admin (non-Wikispooks members) or use this discussion page (Wikispooks members)
Wikispooks logo.png Policy.png Wikispooks to do.png Tools2.png
Wikispooks
Technical
SMW.png Help.png