History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: XCF-2044
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Raghu Venkataramana
Reporter: Martin Steinmann
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
sipXconfig

Allow backup / restore to and from an FTP server

Created: 2007-12-03 11:41   Updated: 2008-10-08 07:46
Component/s: None
Affects Version/s: 4.0
Fix Version/s: 3.11.5

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
File Attachments: 1. Text File XCF-2044-rebased.patch (85 kb)
2. Text File XCF-2044-updatedHelp.patch (3 kb)
3. Text File XCF-2044-updateUI.patch (49 kb)
4. Text File XCF-2044.patch (90 kb)

Image Attachments:

1. backup.png
(86 kb)

2. backupFtpHide.png
(130 kb)

3. backupFtpShow.png
(134 kb)

4. backupLocal.png
(128 kb)

5. Configuration.png
(83 kb)

6. restore.png
(84 kb)

7. RestoreFtpHidden.png
(100 kb)

8. restoreFtpShow.png
(108 kb)

9. RestoreLocal.png
(103 kb)
Environment: 3.9
Issue Links:
Dependancy
 
This issue required for:
XCF-2639 FTP backup should allow for specifyin... Major Open
Related
This issue related to:
XCF-2308 add UI for mounting network shares fo... Major Open
 


 Description  « Hide
In addition to storing backups on the same system or sending them by email, it should be possible to configure an FTP server somewhere on the network to serve as the backup server. Backups and restore operations should be automatic from that FTP server.

 All   Comments   Work Log   Change History      Sort Order:
Mircea Carasel - 2008-02-28 08:33
As a solution proposal, I am thinking to add a new configuration page where the user can configure the FTP server where backup / restore will be handled (Configuration.png).
This page can be accesed from both restore page and backup page (restore.png and backup.png) - user clicks on the "Configuration" Quick Link.
When Local is used then backup or restore will work as currently (local configured path will be used: var/sipxdata/backup).
When FTP server is configured, then the backup or restore will use the ftp server as a configured path.

I am going to send an email on the list with this proposal.
Please let me know what do you think.

Martin Steinmann - 2008-02-28 08:43
Yes, I think this looks very nice.

- Are you saying that backup to local would be turned off if FTP is selected? Why wouldn't we backup to local always and as a default?
- The FTP server IP address can also be a hostname
- How would you select which backup from the FTP server should be restored?
- What would happen if a backup to FTP is scheduled but at the time the FTP server is unavailable?
- Would the backup mechanism notify the admin of successful backups?

Martin Steinmann - 2008-02-28 12:02
Adding Microsoft support would be a great added benefit (XCF-2308)

Mircea Carasel - 2008-04-23 09:52 - edited
The patch file is: XCF-2044.patch
I tried to design this feature to make it flexible - it can be easily extended for more than one ftp server in future.
The fix consists of:
1. There are two backup plans now: the local backup plan and the ftp backup plan
     You can schedule both plans to backup data at a given time: on local and on the configured ftp server.
     Also you can provide different email addresses for sending the configure archive.
2. Both Backup page and Restore page have added a checkbox - to switch from ftp to local any time
     The ftp checkbox setting is saved in the user session and is not persisted in database.
3. A new page is added: BackupRestoreConfiguration. The user has access to this page from Backup Page or from Restore page just by clicking a quick link.
     Here the user configures the FTP server credentials that are saved in the database.
4. There is a new backup directory created: ftpBackup. Here it is stored backup data that is uploaded on the
    FTP server. The content of this directory is deleted every time a FTP backup is issued. In this way the user is able to download the backup data on his machine just in the same way as for local backup.

Also : there is a new jar needed for FTP server client management: commons-net-1.4.1.jar

Mircea Carasel - 2008-04-24 03:28
New library required for this patch: commons-net

Home page of the project: http://commons.apache.org/net/
Version number: 1.4.1
License type: Apache License Version 2.0 : http://commons.apache.org/license.html

Damian Krzeminski - 2008-05-23 11:58
I'd like to finish clean up for XCF-2557 before I take it.

Kevin Thorley - 2008-06-10 18:19 - edited
Mircea, I started working on taking this patch, but there are quite a few conflicts when I try to rebase it against the current code. I applied it to a revision from the day you submitted it, then tried rebasing that to use some mergetools, but even that was daunting. Would you mind doing a rebase against the current code, since you are far more familiar with the BackupPlan code than I am? In the meantime, I will work on getting the commons-net jar setup to use an rpm dependency. Thanks!

Kevin Thorley - 2008-06-11 10:23
Please see my previous comment re: rebasing

Mircea Carasel - 2008-06-11 10:26
I almost finished rebasing with the latest code. I will post a new patch today.
Thanks, Mircea

Mircea Carasel - 2008-06-11 11:58
Please find attached a rebased patch for this issue:
XCF-2044-rebased.patch

This patch is based on the original XCF-2044.patch - it contains only some changes needed in order to align this patch with the latest version of sipXconfig code.


Kevin Thorley - 2008-06-11 12:48
Thanks Mircea, I'll have a look at this now

Kevin Thorley - 2008-06-12 10:27 - edited
Mircea, overall the patch looks good. I plan to commit it this afternoon, as I am leaving on a trip and want to get it in for people to play with. However, I am not closing the issue as there are some things that I think need to be addressed:

1) The admin needs to be able to define a remote backup directory. The remote server probably does not have the same directory structure as the sipX server. I was able to modify the sipxconfig.properties file to specify the backup dir as ~/ on the remote machine, and then I was able to do a backup, but this needs to be exposed to the admin in the ui

2) I am confused about the links that appear on the page after the backup is complete. Why do we allow the admin to download a remote backup from the local system? Maybe this is a good thing, I'm not sure, but at least it should specifically say something about downloading a copy of the backup from the local system... I assume it would go to the remote system to do the download.

3) The help text needs to be updated to provide more info on ftp backup.

Please submit a patch on top of the patch that you have already submitted. I had to make some changes related to the commons-net lib and how we add it to the classpath, so it will be easier just to apply the smaller patch with the above-mentioned changes.

Thanks!

Mircea Carasel - 2008-06-12 14:19 - edited
Kevin, please find below my comments on your questions:
1) - When backup is performed it is uploaded on the ftp server on the location that is exposed when ftp username/password is supplied. Depending on what username/password is provided by accessing the Backup/Restore Ftp Configuration page the backups will be uploaded in that location.
For instance the administrator can configure the FTP server with how many user he wants - each user having therefore a coresponding uploading location
The location that is specified in the sipxconfig.properties - ftpBackup - is used in the ftp backup process like this:
First the files are temporary saved on local sistem in the ftpBackup directory (located in the same path as the "backup" directory -> /usr/local/sipx/var/sipxdata)
and then are uploaded on the ftp server in the location exposed by the username/password provided in the FTP Backup/Restore Configuration page
When the user issues another ftp backup operation the previous temporary savings are deleted

So my opinion is that in fact the sipXconfig should not be aware of the remote ftp path where the backup is uploaded - this path is automatically exposed with the FTP connection, depending on the username/password supplied. So the administrator shoud configure the FTP server itself with what users he wants.
In this way the Backup/Restore is easy to handle - we can have one ftp server with many users and therefore many remote ftp upload paths.
We can pick any path we want just by providing the coresponding username/password


2) The links that appear on the page after the ftp backup is completed exposes the temporary backups located on local system (in the ftpBackup directory described above) - so the user to immediately download them if he wants. Of course the user will find them also uploaded in the specified ftp server in the remote location defined by the provided username/password
I added this comments on the list
Please see: http://list.sipfoundry.org/archive/sipx-dev/msg11935.html
Based on discussions there is a new jira issue that requires this one and addresses the admin ftp backup directory configuration from sipXconfig: XCF-2639

Kevin Thorley - 2008-06-12 14:21
Committed initial changes in r12869

Damian Krzeminski - 2008-06-13 10:36
Need to build jakarta-commons-net dependency for distros where it's not available

Mircea Carasel - 2008-06-16 03:11
I will assign this issue back to me and I will provide a new patch that will address points 2 and 3 from Kevin's original comment
2 - links will remain on the page and I will add specific comments about the purpose of them
3- will update help text

Point 1 will be addressed by XCF-2639

Damian Krzeminski - 2008-06-16 09:00
I did not realize Mircea wanted to keep this open: sorry.
I recommitted initial patch since we are now building jakarta-commons-net for platforms that do not have it.

Mircea Carasel - 2008-06-16 10:30
New patch added as per Kevin's request. This patch updates Backup/Restore help text in order to address Kevin's requests 2 and 3.

Martin Steinmann - 2008-06-16 15:10 - edited
This comment relates to the current code in mainline and does not include the most recent patch. It seems to me that the user interface still deserves some usability improvements. Let me try and summarize my comment as follows:

1. The FTP checkbox at the top looks a bit lost. I am proposing a section (on the same page with some separation) that allows the configuration of backup options. We currently have 3 options and might have more in the future. They are: a) FTP, b) email, and c) local. All three should be configurable independently where each one can be enabled or disabled separately. Parameters like how many backups to keep, email address or the FTP server configuration should be part of the configuration section for the respective backup option. I don't like the link on the right side that takes you to the FTP options - it is too much hidden away.

2. A second section (on the same page with some separation) should allow the configuration of what gets backed up (i.e. configuration and voicemail). Some help text underneath the options explains what these parameters mean

3. Scheduling should be a separate tab (left side navigation). I assume scheduling applies to all the different backup options.

4. Backup now button: There should be such a button next to every backup option so that it is clear when you select Backup Now what kind of backup is started.

5. Restore: Restore should bot be a separate top level page but instead a tab under backups. The restore page needs to be cleaned up so that it is clear through what mechanism a backup is restored. Right now this page is rather confusing to me, especially since we added the FTP checkbox at the top.

6. Help text. The help text needs to be updated according to the suggested changes above.

Damian Krzeminski - 2008-06-17 17:14
QuickHelp patch is committed:

http://sipxecs.sipfoundry.org/ViewVC/sipXecs?view=rev&rev=12898

XCF-2044 quick help updates for FTP backup


I am keeping the issue open for now since we have a discussion on the sipx-dev list.

Al Campbell - 2008-07-08 15:21
Please update based on Martin's new comments.


Mircea Carasel - 2008-07-24 10:47 - edited
XCF-2044-updateUI.patch consists of:

-common/AjaxToggleButton (new component) - new Button that has the function to show/hide a set of blocks (ajax based)
                         - 3 parameters: linkShowText/linkHideText (the button's labels for show/hide)
                                                     renderBlocks (the blocks to show/hide)
-BackupPage/RestorePage now contains a combo box with backup types
-make use of common/AjaxToggleButton in order to show/hide FTP settings for the FTP backup plan type - as requested
-removed FTP checkbox and Configuration Link
-BackupRestoreConfigurationPage is now removed and replaced by BackupRestoreConfigurationPanel component that contains the FTP setting widgets
-tests added for toggling (but commented since current version on http unit does not support Ajax/DOJO capabilities)
-tests added/changed to reflect new UI change for Backup/Restore

Mircea Carasel - 2008-07-24 10:53
Screenshots added with UI change requested/implemented:
1.BackupLocal.png
2.BackupFtpHide.png
3.BackupFtpShow.png
4.RestoreLocal.png
5.RestoreFtpHide.png
6.RestoreFtpShow.png


Damian Krzeminski - 2008-09-02 15:35
http://sipxecs.sipfoundry.org/ViewVC/sipXecs?view=rev&rev=13295

XCF-2044 improved UI for FTP backup/restore

http://sipxecs.sipfoundry.org/ViewVC/sipXecs?view=rev&rev=13296

XCF-2044 stream-line quick help text on backup page

chitralekha.S.H - 2008-10-08 07:46
This issue is resolved.All the ConfigUI pages are implementd.
Issues with respect to this feature will be tracked separately.

Thanks,
Chitralekha.