Setting up Svn Repository over apache2 easy and quick:
Step1>Downloading and Installing Svn modules and dependencies :
Download the most recent distribution tarball from the below mentioned URL :
http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=260&expandFolder=74
Subversion depends on a number of third party tools and libraries.The Subversion has a package containing the minimal prerequisite libraries (Apache Portable Runtime, serf,neon,sqlite-amalgamation and Zlib) called the “dependency package” tarball or zipfile.
-
You can simply unpack the dependency package “on top of” the Subversion package; for example, if you are using a .tar.gz bundle on Unix, you could type: $ tar xzvf subversion-1.x.x.tar.gz $ tar xzvf subversion-deps-1.x.x.tar.gz $ cd subversion-1.x.x
-
and then use the standard GNU procedure to compile: -
$ ./configure - On Unix systems, the ‘./configure’ script will tell you if you are missing the correct version of any of the required libraries or tools. (We can use the said command with prefix –enable maintainer mode or other prefixes as desired.)
-
$ make
-
$ make install
-
Test and verify the above setup by running “make check” at the end.
Step1(a)> Building a Subversion Server:
Subversion has two servers you can choose from:svnserve and Apache.
-
Svnserve is a small, lightweight server program that is automatically compiled when you build Subversion’s source.
-
Apache is a more heavyweight HTTP server, but tends to have more available features which help us give a better and secure control over the Repository.
Step2> Downloading and Installing Apache2 :
Please refer to the below mentioned URL for the latest released version:
http://httpd.apache.org/download.cgi
Compile the below mentioned configuration files as said below:
$ ./buildconf
$ ./configure –enable-maintainer-mode (This arg says to include debugging information.As similar we did while running the ./configure for svn as said in above process.Same setup should be followed else we can run into issues .Take this is as a feedback keep the prefixes usage as same as you do for svn & apache modules).
Compile and install apache:
$ make
$ make install
Step2(a)> Making and Installing the Subversion Apache Server Module :
Go back into your subversion working copy and run ./autogen.sh if you need to. Then, assuming Apache httpd 2.0 is installed in the standard location, run:
$ ./configure
Note: do *not* configure subversion with “–disable-shared”! mod_dav_svn *must* be built as a shared library, and it will look for other libsvn_*.so libraries on your system.
Note: If you have old subversion libraries sitting on your system,libtool will link them instead of the `fresh’ ones in your tree.Remove them before building subversion.
$ make clean && make && make install
Step 3> Configuring and authorizing Apache for Subversion:
By default you will find the httpd.conf file at the below mentioned location:
> /usr/local/apache2/conf/httpd.conf
Verify the below svn modules are added in your httpd.conf file :
LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so
Note: Create a file with the name as ’svn-auth-file’ and save it under /etc directory.
Add the below mentioned svn module to the end of the httpd.conf file. This is one of the basic example for setting up your Svn repository with limited access.
<Location /repos>
DAV svn
SVNPath /absolute/path/to/repository
AuthType Basic
AuthName “Suvbersion Repository”
AuthUserFile /etc/svn-auth-file
Require valid-user
</Location>
Please refer to the below URL for other authorization options available:
http://svnbook.red-bean.com/en/1.0/svn-book.html#svn-ch-6-sect-4.4
Step 3(a)> How to add a SVN user in svn-auth-file :
1.To do this use command htpasswd -m /etc/svn-auth-file “UserName”
(Here the -m option modifies the svn-auth-file)
2.This will add a user and prompt for a password
3.Enter a desired password
4.Restart the apache server with command :/etc/init.d/httpd restart or
sudo /etc/init.d/apache2 restart
5.To delete the user . Use command htpasswd -D /etc/svn-auth-file userName
(Here the -D option delete the desired user from the svn-auth-file)
Step 4> Running the apace server with Svn plugin :
Trigger the below mentioned command:
$ /usr/local/apache2/bin/apachectl start or /etc/init.d/apache2 start
(This will start your apache server with effect to the recent modifications done in httpd.conf file.)
$ /usr/local/apache2/bin/apachectl stop or /etc/init.d/apache2 stop
(This command stops the apache server and you can re-edit your httpd.conf as and when desired and re-start your apache server to have the changes into effect).
Check /usr/local/apache2/logs/error_log to make sure it started up okay.
Step 5> Testing and using the SVN repository:
Steps to checkout and checkin some stuff to your repository :
$ svn checkout/co http://localhost/svn/repos (The most common reason this might fail due, to the permissions for reading the files from the repo DB.Make sure that the httpd process has permission to read and write to the repository).
$ svn commit/ci - This command recursively sends your changes to the SVN server. It will commit changed files, added files, and deleted files. The -m option should always be used to pass a log message to the command.
$ svn add - When you are creating a new file or directory, you need to tell the SVN server about it. This command authenticates the newly created file or directory to the SVN server.
$ svn update/up - This command syncs your local sand box with the server. If you have made local changes, it will try and merge any changes on the server with your changes on your machine.
$svn help - Provides a summary of the available commands.
Note: Please proceed to Step3 If you already have a SVN server installed on your Unix machine. If you are starting afresh ,follow the steps from the beginning.
As per my understanding I have tried to explain the setup with easy and quickly understood steps.Hope this might help to the extent of setting up your client Repository quick and easy with minimal configuration issues.If I am missing something here and if there are any valuable feedback feel free to say them.
Cloud musings!
Suddenly, everyone is looking for fuel efficient cars. That is what is happening with IT resources as well. Cloud is enabling that to happen in a very rapid and scalable manner. Over the last decade, there has been a real push to look at IT services from the service centric point of view. Businesses that have done this were able to innovate faster and provide best value to their customers.
Cloud computing has graduated and has really come out of clouds. It re-affirms the benefits of thinking at a service centric manner. Cloud computing takes the IT services to the next level by virtualizing almost everything that IT provides. A variety of Cloud services has come out of the box. Cloud providers can now provide storage, cpu and network out of the box for a fees.
Cloud could also be a threat to traditional models of software licensing. What will be the impact of Cloud computing on software development, deployment, support and maintenance?
Can a warehouse application be deployed on Cloud and offer the same level of customization and SLA’s for different customers.
Cloud has some more challenges to face. Cloud portability, security and compliance are the keys areas innovation has to happen in future to take cloud to the main stream. Location of the data in various geographies and data disposal could be a compliance issue for sensitive information.
Biggest beneficiaries and customers for the cloud are governments. Governments have huge data that needs to be stored and safeguarded. Security is one man aspect of the cloud. Governments would probably go to private clouds to optimize the resources and thereby reducing operating costs.
ISV’s and ERP companies have already embraced cloud computing and are using for their main line of businesses. Companies that need computing power have used Cloud and are using it on a pay as you go basis and saving huge costs. Cloud will make innovations in the business and could reduce barriers of entry for some businesses to get into the space, which they could not get to earlier.
Hacking in cloud. Can some one who is smart gets onto Cloud computing, really wipes all your machines, data and not even leave traces of what happened?. Auditing and control could be one of the main compliance issues as well.
Applications need to gel with cloud infrastructure seamlessly. We know for sure, how much time and energy we spent in starting a new production release, even of the machines and operating system remains same. Ability to simulate cloud environments locally and deploy in cloud infrastructure seamlessly is challenge.
At Imaginea, we think, Cloud has to be embraced in a step by step manner rather than a big bang. For service companies, there is also a point, where they will start paying more for cloud services compared when hosted in house.
Deploying & hosting application on Amazon-ec2 Unix environment.
Amazon ec2 presents a true virtual computing environment, allowing you to use web service interfaces to requisition machines for use, load them with your custom application environment, manage your network’s access permissions, and run your image using as many or few systems as you desire.
Pre-requisites :
1)Account for amazon S3 and ec2 .
2)Command-line tool (ec2-ami-tool).The AMI Tools are a set of command line utilities for creating,bundling and uploading AMIs to S3 storage where they can be used by ec2.
a). Download ec2-ami-tool from the below mentioned URL:
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351
Select rpm for linux and zip for windows.
b). Installation : To install the AMI Tools unzip the installation file into a suitable installation directory
such as ‘/usr/local’:
unzip ec2-ami-tools-X.X-XXXX.zip -d
This will create the directory ‘/ec2-ami-tools-X.X-XXXX’
where X.X-XXXX represents the release and build numbers.
Before running the utilities the ec2_HOME environment variable needs to be set:
export ec2_HOME=/ec2-ami-tools-X.X-XXXX
Running and authorizing an instance :
1). Sign up for Amazon S3 & ec2 .
2). Create a work directory and there download and unzip Command-Line Tools(ec2-ami-tools.zip) .
3). Create a new X.509 Certificate to generate a PEM encoded signed X.509 certificate (cert-*.pem) and an unencrypted, PEM encoded RSA private key that corresponds to the X.509 certificate (pk-*.pem) and save the certificate and private key in the work directory.
4). Setup environment variables by creating a batch file with the following configuration :
set JAVA_HOME =
set EXTRACT_DIR =
set ec2_HOME = \
set PATH = \bin
set ec2_PRIVATE_KEY = \PRIVATE_KEY_NAME
set ec2_CERT = \CAERTIFICATE_KEY_NAME
copy this batch file onto your work directory and execute it.
5). Run the command > “ec2-describe-images” : it will return a list of available images.
(on a mac you must set JAVA_HOME by doing: export JAVA_HOME=/home/foobar/jdk1.6.0 )
Output will be something like this :
IMAGE ami-68ae4b01 ec2-public-images/fedora-core4-base.manifest 206029621532 available public
Description of the above said command :
Record Type Identifier - IMAGE
Image Identifier - ami-68ae4b01
Manifest Location - ec2-public-images/fedora-core4-base.manifest
User Identifier(Who has registered the image) - 206029621532
Image Status - available
Visibility - public
6). Create a Keypair : “ec2-add-keypair ”
In order to launch an AMI we need a named key pair. The name of the key pair is specified in the web service call that launches the instance and the private key is used by ssh to authenticate for the same.
Use this private key that is hence created, save it to a file called id_rsa-gsg-keypair.
Now set it’s permissions to be rw——- (chmod 600 id_rsa-gsg-keypair).And copy this keypair file onto your work directory
7). Create a group : “ec2-add-group group_name -d ‘description_of_group’ ”
Security is important for any machine hooked up to any network at any time .Amazon’s ec2 cloud has a built-in firewall capability that is easy to setup. It is based on the concept of security groups. A security group is simply a group of access rules that apply to any instance which is a member of that group.
8). Create and Authorize a Security group to HTTP :
Follow the below mentioned commands for opening specific ports for ssh & http respectively.
ec2-authorize default -p 22 (ssh)
ec2-authorize default -p 80 (and http)
9). Now we have done the basics, here is the final step to get an instance up and running on Cloud.
Create an instance of your machine : ec2-run-instances ami-xxxxxxxx -k gsg-keypair
Where ami-xxxxxxxxx is the id of the machine you used to as in step 5 with ec2-describe-images.If this step failed and reported creation of the instance something like this as seen below :
So output will something like this :
INSTANCE i-xxxxxxxx ami-xxxxxxxx gsg-keypair
Description of the o/p of the above command :
Output Type Identifier - INSTANCE
Instance ID - i-xxxxxxxx
AMI ID - ami-xxxxxxxx
Key name - gsg-keypair
Hence ,wait for it,
Using ec2-describe-instances you can check the status of your instance, it will display the desired details as mentioned below:
INSTANCE i-xxxxxxxx ami-xxxxxxxx domU-##-##-##-##-##-##.usma1.compute.amazonaws.com running gsg-keypair
Description of the o/p of the above command :
Output Type Identifier - INSTANCE
Instance ID - i-xxxxxxxx
AMI ID - ami-xxxxxxxx
Public DNS name - domU-##-##-##-##-##-##.usma1.compute.amazonaws.com
Instance state - running
Key name - gsg-keypair
10). Url to access your application installed on ec2 instance is : http://publicDNSname/appname
For example : http://domU-##-##-##-##-##-##.usma1.compute.amazonaws.com/mworks
You can ssh into the machine using the key you created in step 6
ssh -i id_rsa-gsg-keypair root@domU-##-##-##-##-##-##.usma1.compute.amazonaws.com
and user will view the o/p as seen below :
__| __|_ )
_| ( /
___|\___|___|
Welcome to an ec2 Public Image 
S3(Simple storage service) steps & configuration :
How to take the backup of your hosted instance(s) as an Private Image for storing it on Amazon S3(Simple Storage Service)
Taking backup of instance means storing the complete instance as an image in S3(Simple Storage Service),for this first we need to create a folder called ec2 on your instance with the following details:
1) ec2-api-tools-1.2-13740
2) keypair_name.ppk
3) keypair_file
4) setup.sh(batch file for setting the Environment variables)
5) certificate
6) private key
one more batch file Setup.sh is required(For bundling up the instance as an image and upload onto S3 in a Bucket) .
Setup.sh should contain following details:
#!bin/sh
export JAVA_HOME = java_home_location
export EXTRACT_DIR = location_of_ec2_folder
export ec2_HOME = EXTRACT_DIR/ec2-api-tools-1.2-13740
export PATH = ec2_HOME/bin
export ec2_PRIVATE_KEY = EXTRACT_DIR/private-key
export ec2_CERT = EXTRACT_DIR/certificate
After setting up the environment variables , provide all permissions using this command - chmod +x setup.sh .And then execute this .sh file as mentioned below:
> ./setup.sh.
If it doesn’t execute properly then it may be possible that it is not in proper unix format, hence execute the below mentioned command:
> dos2unix setup.sh setup.sh
the above said command formats your .sh file in proper unix format.Now we can execute all ec2-api commands hereafter on the hosted instance.
We need one more called jSh3ll(The jSh3ll is a Java based command shell for managing your Amazon S3 objects) on the hosted instance. And one more batch_file.sh will contain following details:
#!/bin/sh
echo “Starting to script”
current_date=`date ‘+%Y-%m-%d-%H-%M-’`
export JAVA_HOME = jdk location
export EXTRACT_DIR = location_of_ec2_folder
export ec2_HOM = EXTRACT_DIR/ec2-api-tools-1.2-13740
export PATH = ec2_HOME/bin/JAVA_HOME/bin
export ec2_PRIVATE_KEY= EXTRACT_DIR/private_key
export ec2_CERT = EXTRACT_DIR/certificate
export jShellDir=/usr/ami/jSh3ll/(location of jSh3ll)
s3Dir=$jShellDir$current_date
s3inputCommands=$s3Dir”s3commands.txt”
s3output=$s3Dir”s3output.txt”
echo “bucket “$current_date”myappS(bucket_name)” > $s3inputCommands
echo “createbucket private” >> $s3inputCommands
echo “quit” >> $s3inputCommands
cat $s3inputCommands
$JAVA_HOME/bin/java -jar $jShellDir/dist/jSh3ll.jar -h s3.amazonaws.com -u AWS_ACCESS_KEY -p AWS_SECRET ACCESS KEY -i $s3inputCommands -o
$s3Dir”s3output.txt”
echo “create bucket output”
cat $s3Dir”s3output.txt”
echo “after creating bucket: “‘date’
currentDir=/usr/ami/
amiDir=/mnt/backup/
#mkdir “/usr/ami/backups/”$current_date
dbDir=”/usr/ami/backups/”$current_date”/mysql”
#mkdir $dbDir
mkdir “/mnt/backup/”$current_date
/usr/local/bin/ec2-bundle-vol -r i386 -d “/mnt/backup/”$current_date -c /usr/ami/ec2/CERITIFICATE_NAME -k /usr/ami/ec2/PRIVATE_KEY_NAME -u ec2
_USERID -a -p $current_date”myapp(BUCKET_NAME)”
echo “after bundling: “`date`
/usr/local/bin/ec2-upload-bundle -b $current_date”myapp” -m “/mnt/backup/”$current_date”/”$current_date”myapp.manifest.xml” -a AWS_ACCESS_KEY -s
AWS_SECRET ACCESS KEY
echo “after uploading: “`date
ec2-register $current_date”myapp”/$current_date”myapp.manifest.xml”//This command register whole data as an image(private)
This script will bundle the instance as an image and upload it in S3 in a BUCKET with unique name.
One great thing about Amazon S3 is that all uploaded files are available as URLs from a web browser (or any application that can read a URL). The format of the URL is as follows:
http://.s3.amazonaws.com/
Recovering data from S3 :
For this we should go to jetS3t directory there we will see configs/synchronize.properties.Provide AWS
ACCESS KEY ,JAVA_HOME and AWS SECRET ACCESS KEY,something like this in it.
####################################
# Synchronize application properties
#
# This file must be available on the
# classpath when Synchronize is run
####################################
# AWS Access Key (required)
accesskey=01G4VJZ4VPV73ZXV10G2
# AWS Secret Key (required)
secretkey=5hl8jETZD7wvAsS5VDuQgfiTyuafExGgKbbGFRE4
export JAVA_HOME=/usr/jdk/jdk1.6.0_04
# Access Control List setting to apply to uploads, must be one of: PRIVATE, PUBLIC_READ,
PUBLIC_READ_WRITE
# The ACL setting defaults to PRIVATE if this setting is missing.
acl=PRIVATE
# Password used when encrypting/decrypting files. Only required when the –crypto setting is used.
# password=
Create a folder in which you want to store the data,after this go to jetS3t/bin directory and execute the below command.
> ./synchronize.sh –keepfiles DOWN bucketname foldername.
Now the data is being stored in the desired folder.When you will execute ec2-describe-images then will return back with the private image which user has created and using the desired image we can easily launch/host a new instance.
Useful Link:http://jets3t.s3.amazonaws.com/applications/synchronize.html