WebSockets overcomes the above problem by providing full-duplex bidirectional connection between client and server which enables servers to send data to client without the need for client to request for it. Few examples where WebSockets are useful are chat, website displaying stock market prices, website showing game scores etc.

In this article, let us try to implement a simple group chat application to demonstrate WebSockets. We will be using Mojolicious Web Framework here. Mojolicious has WebSockets support by default which makes it quite easy to implement this.

We will keep our chat application simple with the below features:

• Users should be able provide their name and connect to the chat application
• Messages sent by any user should be available to everyone connected to the chat application
• Users inactive for 5 mins should be automatically disconnected from the chat application

Server-Side Logic

For the purpose of this demonstration, I will use Mojolicious lite app. Let us go ahead and create our application using the below command:

mojo generate lite_app simpleChatApp

Let us now write a route to handle any websocket requests. In Mojolicious you use the websocket request to do that. We will also set a timeout of 5mins so that any user not active for more than 5mins gets automatically disconnected. We will store all the connected user information in a hash(%users) so that we can broadcast any new message to everyone.

Once the user is connected, any new message sent by any user needs to be broadcasted to all the users that are there in the %users hash. We will parse the pipe delimited message received from the user which will have the name and the message(we will add this logic in the HTML template in the next section). We will append the current time and send all the information in JSON format to all the connected users (%users).

If the users disconnects or time-out occurs, we will remove the user from our hash so that new messages are not sent to them.

Client-Side Logic

Now that we have our server side logic in place, time to write client side logic. Javascript has APIs which can be used to connect to server via websockets. We will be using that here.

First, we will have to connect to server via websocket either when the page loads or on some user action. In our case, we will give a button to the user to connect to the server by providing his name; when that button is clicked, we call the below code to connect to the webserver and display a message saying Client Connected. We have HTML elements to display the messages and capture the user inputs. You can check the complete source code here.

Once the connection is established, whenever the user sends a new message, we append his name with the message he has typed and send it to the server delimited by pipe(you should ideally be using JSON or something!).

When anyone sends a message (including the user himself), the message which is broadcasted by the server will be captured, parsed and displayed on the chat window. Chat window is a HTML text area, you can check the source code here.

In case the user is inactive for 5mins or the server closes the connection, we pop-up a message to the user saying connection is closed:

Testing

Would you believe if I told you, this is all you need to build a simple chat program? Well, that’s Mojolicious for you. Let us fire up the morbo server and check out our chat application.

Let us try accessing our chat program using the url: http://127.0.0.1:3000

Give your name and click Connect button to connect to the chat session.

A welcome message is displayed and it also shows that the user is connected to the server. Now go ahead, enter a message and hit send.

You can see your message in the text area with a time stamp. The message is actually broadcasted by the server to all the connected users. Now let us access this chat program from our phone and send a message.

You can see the messages sent by any user getting broadcasted to all the active users. If a time out occurs or if the server disconnects the connection, all the users will get a pop-up message like this:

That’s about it, we have successfully implemented a simple chat program in Mojolicious using websockets. You can find the complete source code of this program here. Below is the video demonstration of what we discussed in this article, if it interests you. Thank you for reading.

The post WebSockets using Mojolicious appeared first on The Curious Technoid.

Minion is a high performance job queue for the Perl programming language, with support for multiple named queues, priorities, high priority fast lane, delayed jobs, job dependencies, job progress, job results, retries with backoff, rate limiting, unique jobs, expiring jobs, statistics, distributed workers, parallel processing, autoscaling, remote control, Mojolicious admin ui, resource leak protection and multiple backends (such as PostgreSQL).

mojolicious.org

Minions doesn’t come pre-installed with Mojolicious installation so it needs to be installed separately.

cpanm Minion

If you use docker, I have built an image which has Mojolicious and Minion together. You can get it here:

docker pull curioustechnoid/mojolicious

Minion uses database backend to store and process the jobs. Most of the popular databases(like SQLite, PostgreSQL) are supported. Jobs can be added to the queue from different programs like Mojolicious, Perl Program, Command Line etc. The Minion worker will be monitoring your backend database and processes the jobs based on priority and availability.

There are 2 important components to Minions:

• Minion Worker
• Job Queues

Let us see how we can configure and deploy Minion.

Minion Worker

Minion workers are the silent warriors that keeps monitoring the backend database and processes the jobs that gets added to the queue.

Let us go ahead and create a script to start the Minion worker. I am using Mojolicious lite app to create a worker. You can very well create a worker without using Mojolicious, read more about it here. We will also add definition of our jobs inside this script. We will be adding a ‘slow-running‘ task and a ‘high priority’ task to the script as an example. For the purpose of this demo, I am using SQLite as the backend.

#!/usr/bin/env perl
#
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# minion-worker.pl
#
# AUTHOR:  Curious Technoid
# DATE:  20-Apr-2021
# VERSION:  1.0
# PURPOSE:
#           This mojo app is to initiate the minion worker
#           which will process the jobs in the queue.
#
# USAGE: 
#       ./minion-worker.pl minion worker -m production
#
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# 
use Mojolicious::Lite -signatures;


# Using sqlite for the queue
plugin Minion => { SQLite => 'minionjobqueue.db', };



#
# Dummy Slow Running Task
#
app->minion->add_task(slow_task => sub ($job) {

        my $id = $job->id;
        $job->app->log->info("Slow Running Task Initiated -> ".$id);
        sleep 20;
        $job->app->log->info("Slow Running Task Completed -> ".$id);
        $job->finish({message => "Slow Running Task Completed -> ".$id});
});

#
# Dummy High Priority Task
#
app->minion->add_task(high_priority => sub ($job) {

        my $id = $job->id;
        $job->app->log->info("High Priority Task Initiated -> ".$id);
        sleep 3;
        $job->app->log->info("High Priority Task Completed -> ".$id);
        $job->finish({message => "High Priority Task Completed -> ".$id});
});



app->start;

Start the worker

Now that our Worker script is ready time to start it. You can manually start the worker by running the one of the below commands in terminal:

./minion-worker.pl minion worker

./minion-worker.pl minion worker -m production

nohup ./minion-worker.pl minion worker -m production &

In a production environment, I prefer creating it as a systemd service and run it in the background.

[Unit]
Description=Minion Workers for Mojolicious
After=mariadb.service

[Service]
Type=simple
ExecStart=/path/to/your/worker/minion-worker.pl minion worker -m production
ExecReload=/path/to/your/worker/minion-worker.pl minion worker -m production
KillMode=process

[Install]
WantedBy=multi-user.target

Now we have the worker running, ready to process any jobs added to the queue. You can check the status of the jobs in the queue using the command:

./minion-worker.pl minion job

Minion comes with a pre-built Admin console to monitor the jobs that are enqueued and processed. It can be enabled using the plugin:

plugin 'Minion::Admin';

We will see it’s usage when we build our Mojolicious app for adding jobs to the queue.

Next task would be create means to enqueue the job queue.

Job Queue

There are many ways to add jobs to the job queue, we will look at the below 3 ways:

• Mojolicious App
• Perl Script
• Command Line

Mojolicious App

One of the best use cases of using Minions is to call it from Mojolicious Web App to do time consuming or CPU intensive task in the background. We will create a Mojolicious App which will do the below things:

1. Add new jobs to the Queue
2. Enable Minion Admin Console to monitor the job queue

Without further adieu let us quickly create a new Mojolicious app to begin with:

mojo generate app mojominion

In our main library created by the mojo command, we will first add the logic to call the Minion plugin.

File: mojominion/lib/mojominion.pm

$self->plugin(Minion => { SQLite => 'minionjobqueue.db'});

We will keep it simple and only add routes to:

1. Add new jobs to the queue
2. Navigate to Minion Admin Console

We will not think much about security as of this moment. This is just to show you how to add jobs to the queue, so right now anybody can add jobs to the queue.

  $r->get('/addjob/slow')->to(controller => 'Example', action => 'addSlowJob');  # Route to add slow job
  $r->get('/addjob/high')->to(controller => 'Example', action => 'addHighJob');  # Route to add high priority job
  $self->plugin('Minion::Admin' => {route => $r->any('/admin')});  # Route to navigate to Minion Admin Console

You can find the complete source code here.

Let us now add the logic in our default controller to push the job to the queue.

File: mojominion/lib/mojominion/Controller/Example.pm

Add the below 2 procedures in the controller:

# Subroutine to add slow jobs
sub addSlowJob ($self) {

   my $jobid = $self->minion->enqueue(slow_task => []);

  $self->render(text => 'Your Slow Task is Added to the queue, Job ID: '.$jobid.'. <a href="/admin">Click here</a> to view the status.');
}



# Subroutine to add high priority jobs
sub addHighJob ($self) {

   # 
   # By default priority is set to 0, anything more than that gets priority.
   # You can have values between -100 to 100
   #
   my $jobid = $self->minion->enqueue(high_priority => [] => {priority => 10});
   # my $jobid = $self->minion->enqueue(slow_task => []); #Another way of adding the task to the job queue

   $self->render(text => 'Your Task is Added to the high priority lane, Job ID: '.$jobid.'. <a href="/admin">Click here</a> to view the status.');

}

Task given high priority will get picked up first once the worker becomes free. You can see this first hand in a video demo that I created in youtube.

You can find the complete source code of the above codes here.

Time to Test

We created the worker, started it and then created a Mojolicious App that can handle job requests. Let us fire it up and see if it works.

morbo mojominion/script/mojominion

Let us first access the admin console to see if that is accessible:

http://127.0.0.1:3000/admin

Admin console is working fine and we can see that we have 1 worker running. Time for us add some jobs by using the below URLs (keep the admin console open):

http://127.0.0.1:3000/addjob/slow
http://127.0.0.1:3000/addjob/high

Open the URL multiple times to add more jobs. If we head back to our admin console we can see our job getting executed.

Just like that we have successfully implemented Minions using Mojolicious.

Perl Script

Can we use Minions without Mojolicious ? Most definitely. We will keep the above Mojolicious app running to access the admin console and then try to add a job using the below plain Perl script:

#!/usr/bin/env perl
#
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# add-minion-job.pl
#
# AUTHOR:  Curious Technoid
# DATE:  20-Apr-2021
# VERSION:  1.0
# PURPOSE:
#           This perl script is used to add a new job to
#           minion queue.
#
# USAGE: 
#       ./add-minion-job.pl slow
#       ./add-minion-job.pl fast
#
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# 
use Minion;


# Do you notice we are using the same database
my $minion = Minion->new( SQLite => 'minionjobqueue.db');


# Getting the parameter
my $param = shift;


if(lc $param eq 'slow'){

    # Add slow task to the queue
    my $jobid = $minion->enqueue(slow_task => []);

    print("Your Slow Task is Added to the queue from perl, Job ID: $jobid\n\n");

}elsif(lc $param eq 'fast'){

    # Add high priority task to the queue
    my $jobid = $minion->enqueue(high_priority => [] => {priority => 10});

    print("Your High Priority Task is Added to the queue from perl, Job ID: $jobid\n\n");
");

}else{

    die("Incorrect parameter\n");
}

Once the script is ready, run it using the below command to add jobs:

./add-minion-job.pl slow
./add-minion-job.pl fast

Now if you head back to the Admin console that is running from our previous example, you can notice that the new jobs added by the Perl script is queued and executed. If you don’t want to use the Mojolicious Admin console at all, you can always use the command line to check the jobs:

./minion-worker.pl minion job

Command Line

We can even add jobs to the queue from command line directly. One way is ofcourse to use the pure Perl script above and use that to add jobs from command line. Not only that, you can also add jobs using the worker script(minion-worker.pl) we wrote in the beginning. Below command adds jobs to the queue directly from command line:

./minion-worker.pl minion job -e slow_task

You can pass arguments like this:

./minion-worker.pl minion job -e slow_task -a '["param1","param2"]'

This is very useful if you want to schedule jobs using cron.

That’s about it. We have configured, deployed and tested Mojolicious Minions in this article. If you wish to check out the hands on demonstration about the things we discussed in this blog, check out my video in youtube below.

You can read more about Minion documentation in the official page here.

The post Mojolicious Minion – A High Performance Job Queue appeared first on The Curious Technoid.

The main functionality of our test case website that we are working on from part 1 of Mojolicious series is for registered users to be able to publish their testimonials in the website. To achieve that, we need to follow the below steps:

• Create Database Components
• Create Testimonials Page Template
• Database Connectivity Logic
• Verification
• Deploy to Production

Create Database Components

The testimonials entered by the users needs to be stored in the database so that all the testimonials entered by all the users is visible to everyone. We will use MariaDB database to store our testimonial table. First, we will create a MariaDB database called tct_mojo_db by logging in as root user:

mysql -u root -p

CREATE database tct_mojo_db;

Then we will create a database user called demo and give it all the privileges to the database tct_mojo_db

CREATE USER 'demo'@'localhost' IDENTIFIED BY 'welcome123';

GRANT ALL PRIVILEGES ON tct_mojo_db.* TO 'demo'@'localhost';

Finally we login as the new user(demo) and create the table tct_mojo_testimonials in our new database tct_mojo_db

mysql -u demo -p

use tct_mojo_db;

CREATE TABLE IF NOT EXISTS tct_mojo_testimonials (
id             INT(20)         NOT NULL AUTO_INCREMENT,
published_by   VARCHAR(560)    NOT NULL,
published_on   TIMESTAMP       NOT NULL DEFAULT CURRENT_TIMESTAMP,
testimonial    VARCHAR(32000)  NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB;

We will store the testimonials published by the users in the above database table. We have all the database related components in place for your project.

Create Testimonials Page Template

We need a new page template where users can publish their testimonials and view the testimonials of others. We will have a single page which does both. The first section of the page will be a html form which registered users can use to provide their testimonials and we will have a section below which will have all the testimonials given by everyone.

Let us go ahead and create a new template called: managetestimonials.html.ep in our templates folder: myWebSite/templates/myTemplates/ with the below content:

% layout 'master';
% title 'View Testimonials';
<a href="/">Home</a>&nbsp;&nbsp;
<a href="/logout">Log Out</a>

<h1><%= $msg %></h1>

<center>
<form action="/testimonials" method="post">

    <textarea id="userReview" name="userReview" rows="8" cols="50">Please enter your testimonial here</textarea></br>
    <input type="reset" value="Clear" />
    <input type="submit" value="Publish">

</form>
</center>

<table width=100% align=center border=1 cellspacing=0 cellpadding=0>
  <tr>
    <th>Testimonials</th>
  </tr>

  <%== $alltestimonials %>

</table>
</body>
</html>

We are referring to a perl variable $alltestimonials in the template code, this will have the existing testimonials from database in HTML format. We will set this value in our controller after getting them from database.

Database Connectivity Logic

Now that we have the new page template ready, time for us to add the logic to handle database operations. In order to achieve that we need to make the following changes to our project:

• Update DB credentials in myWebSite.conf file
• Add DB handler and logic to handle new route
• Create DB Modeller for database operations
• Change Controller logic to handle DB operations

myWebSite.conf

We need to update our conf file to include database credentials. We will add the database connectivity in the below format:

mysql => 'mysql://dbusername:dbpassword@localhost/databasename'

So our updated myWebSite.conf file will look like this:

{
  secrets => ['8c285c2b2f9ce11c46ee322d52179ac32be6d42a'],
  mysql => 'mysql://demo:welcome123@localhost/tct_mojo_db'
}

DB Handler

We have DB connections in place. Now let us create a helper in our main library(myWebSite.pm) which will be used as a handle to our database. This can be used my our modeller and controller to access the database.

Database helpers can be created using the below statement:

# Invoking Database handle 
$self->helper(mysql => 
sub { state $mysql = Mojo::mysql->new(shift->config('mysql')) });

$self->helper(dbhandle => 
sub { state $vikidb = myWebSite::Model::Database->new(mysql => shift->mysql) });

We will also add the below routes to handle the new testimonial page action using :

$authorized->get('/testimonials')->to('CustomController#loadTestimonials');
$authorized->post('/testimonials')->to('CustomController#saveTestimonial');

If you remember from our previous session, the $authorized handle helps us in making sure that only logged in users can access the above calls since these statements are called after the under statement.

Do not forget to import the module use Mojo::mysql; in your main library. If Mojo::mysql is not installed, go ahead and install the perl module.

We will also import our database modeller in our main library: use myWebSite::Model::Database;. This does not exist yet, we will be creating this modeller in the next section.

Once all the change are done, your main library file myWebSite.pm should look something like this.

DB Modeller

Next we will be creating a database modeller which will have the logic to query the testimonial data and insert any newly published testimonial. We will create the modeller inside our lib directory using the command:

mkdir lib/myWebSite/Model

touch lib/myWebSite/Model/Database.pm

Let us now open the database modeller file and add the below content:

package myWebSite::Model::Database;
use Mojo::Base -base;

has 'mysql';

# Subroutine to get all the testimonials
sub fetch_all_testimonials{ shift->mysql->db->query('select * from tct_mojo_testimonials order by published_on desc')->hashes->to_array }

# Subroutine to insert the new testimonial to database
sub publish_testimonial{


   my ($self, $new_testimonial, $username) = @_;
   my $sql = 'insert into tct_mojo_testimonials(published_by,testimonial) values (?,?)';


   $self->mysql->db->query($sql, $username, $new_testimonial);

}


1;

We have 2 subroutine in our modeller, fetch_all_testimonials will fetch all the testimonials into an array of hashes and the second subroutine publish_testimonial inserts the newly published testimonials to the database. These are the only 2 database operation we require for your website.

Controller CustomController.pm Changes

We have our modeller in place that handles the DB operations, let us go ahead and add subroutines in our controller which calls these.

In the above section, we had added 2 new subroutine calls to our controller in the main library file: myWebSite.pm. Let us add the logic to those 2 subroutines in our existing controller: lib/myWebSite/Controller/CustomController.pm

loadTestimonials basically generates a HTML data of all the testimonials that are there in the database. The generated HTML data is passed as parameter to the template where it is rendered. It uses the fetch_all_testimonials subroutine from the DB modeller that we created above. In the section Create Testimonials Page Template above we had used the variable $alltestimonials to display the data, the value for it is being set in the below subroutine.

sub loadTestimonials{

    my $self = shift;
    my $all_testimonials_html;

    foreach my $all_testimonials (@{$self->dbhandle->fetch_all_testimonials})
    {
        $all_testimonials_html .= "
  <tr>
      <td>
          ".$all_testimonials->{testimonial}."</br></br>
          <div style='text-align: right;'><i>".$all_testimonials->{published_by}."</br>".$all_testimonials->{published_on}."</i></div>
     </td>
  </tr>";
    }
    $self->render(template => 'myTemplates/managetestimonials',msg => 'View Testimonials',alltestimonials => $all_testimonials_html);


}

saveTestimonial basically saves the newly created testimonial added by the registered users. It uses the publish_testimonial subroutine from the DB modeller that we created above.

sub saveTestimonial{

    my $self = shift;
    my $new_testimonial = $self->param('userReview');
    my $user = $self->session('username');

    $self->dbhandle->publish_testimonial($new_testimonial,$user);
    &loadTestimonials($self);
}

Once all the above changes are done, our CustomController.pm controller will look like this.

Verification

That’s about it, we have everything we need to start our web application. Let us start our web application using morbo webserver:

I have SSH Tunnel enabled, so let me access our website using localhost IP and port 3000.

Once we login and navigate to Testimonials page, it should look like this:

If we add any new testimonial, we can see it getting added in the table below:

We have successfully handled database connection in our website.

Deploy to Production

Now that we have a fully operational website with all the requirements met and tested, we can deploy our changes in production and start it to be available for everyone. To do that, as explained in the Mojolicious Introduction article, we will use the production webserver hypnotoad.

1. Let’s tell hypnotoad to run on port 80. To do that, we will update the myWebSite/myWebSite.conf file to look like this:

{
     secrets => ['8c285c2b2f9ce11c46ee322d52179ac32be6d42a'],
     mysql => 'mysql://demo:welcome123@localhost/tct_mojo_db',
     hypnotoad => {
          listen => ['http://*:80'],
          workers => 10
     }
}

We have also told hypnotoad to run 10 workers.

2. Next we will open firewall ports 80 so that the website is accessible to everyone. I run Mojolicious on Fedora and use firewalld, I will run the below command to open the ports:

firewall-cmd --zone=public --permanent --add-port 80/tcp
firewall-cmd --reload

3. Start the web application using hypnotoad by using the command:

hypnotoad myWebSite/script/myWebSite

4. Finally verify by accessing the website using your server IP (or domain name if you have one mapped):

Note: I recommend to use Apache Reverse Proxy along with hypnotoad for production deployments.

With that the Mojolicious tutorial comes to an end. We have successfully created a fully operational website covering major features of any website and made it available to general public. Hope the Mojolicious tutorial series were informative and helpful. Do let me know your thoughts and comments in the comment section below. You can find the entire source code of the testimonials project here. You can watch the entire Mojolicious series that we discussed in this blog series in my youtube channel. Keep visiting this blog for more technical tutorials.

Edits:

[13-May-2021] It so happened that I missed to mention about changing the homepage.html.ep file to point to the new testimonial route(/testimonials) that we created in the main library(myWebSite.pm). You just need to change the href tag in the homepage template to point to the new route and you should be good to go.

% layout 'master';
% title 'Home Page';
    <a href="/logout">Log Out</a>
      <h1><%= $msg %></h1>
      <h5 class="w3-padding-32">This is my personal site built with Mojolicious. You can provide testimonials using the link <a href="/testimonials">here</a>. 
</h5>

Thanks to Jim for pointing this out.

The post Mojolicious (Part 4): Database Management appeared first on The Curious Technoid.

• Create Login and Logout Pages
• Authentication and Session Management Logic
• Verification

Before we proceed further, let us all remind ourselves how our Mojolicious website looks as of now:

and this is our project file structure:

Create Login and Logout Pages

Instead of the home page, we will redirect the users to the login page whenever they access our website. If they provide the correct username / password, we will create session cookies and re-direct them to the home page. Once they are done browsing our website, they can click the Logout link (which we will add to our homepage). This will remove the session cookies and safely logout.

We have already created a master layout in our previous session which is the skeleton of our website. This makes us not worry about the look and feel of the website as it is already taken care of. We just need to create template files for login and logout pages which will call our already created master layout.

Login Page

Let us create a new template called login.html.ep inside the template folder of our project: myWebSite/templates/myTemplates. We will add the below code to it:

% layout 'master';
% title 'Login Page';

<center>
<h1>Please Login to Access the Website</h1>

<h6>
<!-- Logic to display errors if any -->
<font color=red>
<% if($error_message){ %>
<%= $error_message %>
<% } %>
</font>
</h6>

<form action="/login" method="post">

    <b>UserName</b> <input type="text" name="username" required></br>
    <b>Password</b> <input type="password" name="pass" required></br>

    <input type="submit" value="Submit">
    <input type="reset" value="Reset" />

</form>
</center>

The variable $error_message is used to display any error encountered while validating the user credentials.

We are using the master layout that we created in the previous session. We changed the title helper to say it’s a Login Page. Then we created a very simple login form which on submit is routed to /login action.

Logout Page

We will now create a logout page template with the name logout.html.ep in our template folder: myWebSite/templates/myTemplates with the below content:

% layout 'master';
% title 'Logout Page';

<center>
<h6>You have successfully logged out of the website. Thank you for visiting. If you want to login again, please <a href="/">click here</a></h6>
</center>

Modify Home Page

Let’s also add a logout link to our homepage template that we created in the previous article.

% layout 'master';
% title 'Home Page';
      <a href="/logout">Log Out</a>
      <h1><%= $msg %></h1>
      <h5 class="w3-padding-32">This is my personal site built with Mojolicious. You can provide testimonials using the link <a href="#">here</a>. 
</h5>

Authentication and Session Management

Our login and logout pages are ready, now time to add logic to authenticate the registered users using session cookies. We will create the below 3 subroutines in our controller (CustomController.pm*):

• displayLogin
• validUserCheck
• alreadyLoggedIn
• logout

*myWebSite/lib/myWebSite/Controller/CustomController.pm

displayLogin

This subroutine will display the login page template that we created in the previous section of this article. In case the user is already logged in, they will be redirected to the home page.

sub displayLogin {

    my $self = shift;

    # If already logged in then direct to home page, if not display login page
    if(&alreadyLoggedIn($self)){
    # If you are using Mojolicious v9.25 and above use this if statement as re-rendering is forbidden
    # Thank you @Paul and @Peter for pointing this out.
    # if($self->session('is_auth')){

            &welcome($self);

    }else{

       $self->render(template => "myTemplates/login", error_message =>  "");

    }

}

validUserCheck

This subroutine will check the details entered by the user in the login page and authenticates the user to access the website. This also creates session cookies to keep the user’s session active. The users are prompted with proper error messages when incorrect credentials are passed. We set the variable error_message here which gets displayed in our login.html.ep template.

sub validUserCheck {

    my $self = shift;

    # List of registered users
    my %validUsers = ( "JANE" => "welcome123"
                      ,"JILL" => "welcome234"
                      ,"TOM"  => "welcome345"
                      ,"RAJ"  => "test123"
                      ,"RAM"  => "digitalocean123"
                     );

    # Get the user name and password from the page
    my $user = uc $self->param('username');
    my $password = $self->param('pass');

    # First check if the user exists
    if($validUsers{$user}){

        # Validating the password of the registered user
        if($validUsers{$user} eq $password){

            # Creating session cookies
            $self->session(is_auth => 1);             # set the logged_in flag
            $self->session(username => $user);        # keep a copy of the username
            $self->session(expiration => 600);        # expire this session in 10 minutes if no activity


            # Re-direct to home page
            &welcome($self);

        }else{

            # If password is incorrect, re-direct to login page and then display appropriate message
            $self->render(template => "myTemplates/login", error_message =>  "Invalid password, please try again");
        }

    }else{

        # If user does not exist, re-direct to login page and then display appropriate message
        $self->render(template => "myTemplates/login", error_message =>  "You are not a registered user, please get the hell out of here!");

    }

}

alreadyLoggedIn

This subroutine checks session cookies to see if the user has already logged in, if yes then the user will be automatically allowed to access the pages in the website

sub alreadyLoggedIn {

      my $self = shift;

      # checks if session flag (is_auth) is already set
      return 1 if $self->session('is_auth');


      # If session flag not set re-direct to login page again.
      $self->render(template => "myTemplates/login", error_message =>  "You are not logged in, please login to access this website");

      return;

}

logout

We need to destroy all the session cookies that were created when the user logged in, forcing any visitor to the website to enter the credentials again. logout subroutine does that.

sub logout {

    my $self = shift;

    # Remove session and direct to logout page
    $self->session(expires => 1);  #Kill the Session
    $self->render(template => "myTemplates/logout");

}

After all these changes your controller should look like this.

Now we need to tell Mojolicious to call these subroutine whenever there is any action on the page. What handles the action on the page? If you have gone through the previous articles, you know it’s the main library file: myWebSite/lib/myWebSite.pm. Right now we only have 1 routes in our main library file which looks like this:

# Normal route to controller
$r->get('/')->to('CustomController#welcome');

Lets add the routes to send requests to all the subroutines we created in the previous section

# Normal route to controller
$r->get('/')->to('CustomController#displayLogin');
$r->post('/login')->to('CustomController#validUserCheck');
$r->any('/logout')->to('CustomController#logout');

my $authorized = $r->under('/')->to('CustomController#alreadyLoggedIn');

Our main library file should finally look like this.

Let us just try to understand quickly what all we are trying to do:

$r->get('/')->to('CustomController#login'); Whenever anyone accesses our root path of the website redirect them to login subroutine(which renders login page), instead of the home page which was the case before.

$r->post('/login')->to('CustomController#validUserCheck'); When the credentials are entered and submitted in the login page we will route the request to validUserCheck subroutine which basically validates the user, creates session cookies and redirects the user to website home page.

$r->any('/logout')->to('CustomController#logout'); Whenever Log Out link is clicked, it directs the request to logout subroutine which destroys the session cookies.

my $authorized = $r->under('/')->to('CustomController#alreadyLoggedIn'); This is a special condition. The “under” call ensures nothing after this statement gets executed, if the subroutine in “under” call fails(in our case the subroutine is alreadyLoggedIn). This is especially useful when we have multiple webpages and you want only authorized users has access to it. We will see this is our next session when we build our testimonials page.

Verification

I guess we have done everything that is required to enable authentication to our website. Let us not waste any more time and start our web application with morbo:

When we access our website now, we are greeted with the login page instead of the home page:

If you give incorrect credentials, you will get not be allowed to login and will get the below messages:

Valid users will be logged into the website and they can access the home page. They will also see a logout link in their home page.

With that we have secured our website. This completes the session management and user authentication part of Mojolicious tutorial. Ofcourse in the real world, you would validate the user credentials from your database with encrypted passwords. For the purpose of this demo, I have kept it simple. In the next session, we will cover database management in Mojolicious. You can watch the demo of this article in my youtube channel below.

The post Mojolicious (Part 3): Session Management appeared first on The Curious Technoid.

Layouts are reusable skeleton of your website. This generally would have your header, logo, footer, menus, side bar etc
Templates are the dynamic content that you want to display based on different actions on the web portal. This is generally the content that needs to be inside the layout.

Download Template

To put it into action, let’s download a free HTML template from the internet. I downloaded one from w3 schools.

It looks like this:

You can get the source code here.

Create a Layout

Let us make this HTML template as a layout in Mojolicious, so that we can use it across our pages. To do so, lets follow the below steps:

We will finally have the layout file, master.html.ep which looks like this. You can check comments by The Curious Technoid to know the changes I did to the original file.

This will be our layout which is our website skeleton and will be used across our website. We will only change the content and title dynamically using Mojolicious templates.

Create Templates

Now that we have our website layout in place, we will create templates from where these layouts gets called from. If you have gone through the previous article, you have already created a new template called: myWebSite/templates/myTemplates/homepage.html.ep which looks like this:

% layout 'default';
% title 'Home Page';
<h2><%= $msg %></h2>
<p> This is my personal site built with Mojolicious. You can provide testimonials using the link <a href="#">here</a>. 
</p>

Lets change it to call our newly created layout master.html.ep. We also add our “content” using the same HTML tag and CSS Class which was their in the original HTML source:

% layout 'master';
% title 'Home Page';
<h1><%= $msg %></h1>
<h5 class="w3-padding-32"> This is my personal site built with Mojolicious. You can provide testimonials using the link <a href="#">here</a>. 
</h5>

% layout 'master'; tells Mojolicious which layout to use
% title 'Home Page'; – the title helper which gets replaced in layout
Any HTML data you enter in the template will be displayed in the <%= content %> section of the layout.

Verify Changes

Now that we have our templates and layouts in place. Lets start our web application using morbo:

morbo myWebSite/script/myWebSite

Since we have SSH Tunnel enabled as usual, we will check our website using local host and port 3000:

Our website doesn’t look that bad now, does it.

In the next article, we will make our website accessible by only registered users using sessions and cookies.

The post Mojolicious (Part 2): Layouts and Templates appeared first on The Curious Technoid.

Test Case Synopsis

The test case that we have is to build a web application for testimonials. That’s the easiest one that I could think of to cover the topics I want to discuss. The web application will have the following features:

• Login Page: Which restricts website access to registered users
• Home Page: Main page for the registered users with a welcome message and website instructions
• Testimonial Page: Testimonial Page which displays all the testimonials from database entered by the users and option to save new testimonials in the database
• Log Out: To gracefully logout of the website

We will also add a simple HTML template to the website which would cover the templates and layouts concept in Mojolicious.

Without further adieu, let’s begin building this project.

Create full_app project

In our previous tutorial on Mojolicious, we created “Hello World” web app with lite_app. This time we will build this project with full_app. To create a full_app, open the terminal and enter the below command:

mojo generate app myWebSite

This will create a directory called myWebSite with the below folder structure:

Lets understand the below components from the above screenshot

• lib
• myWebSite.conf
• public
• script
• t
• templates

lib

The library directory is probably the most crucial directory which holds all your programming code. Your Controller and Model logic goes here. The logic to handle any action on your webpage will be somewhere in this directory.

Your routes and page-actions are sent to the file myWebSite/lib/myWebSite.pm which then invokes the relevant Controller and Model to handle your requests.

myWebSite.conf

This is the master configuration file for your web app. You can add secret phrases for signed cookies, mention the number of workers that you want webservers to run, the port that your application should run on, database connection details, so on and so forth.

public

You can find all your static pages here. Along with the static pages, all publicly available components like your css, images or any other media goes into this directory.

script

Script to start your web site is present in this directory. You run you web app using the script/myWebSite file present in this directory.

t

Folder dedicated for your test scripts.

templates

Directory which holds the templates and layouts used by your web app.

I will not go in detail about the theory as you can always read more about it in the official website of Mojolicious. Let us proceed with our project.

Let’s go ahead and run this sample web app as it is, using the development web server morbo:

morbo myWebSite/script/myWebSite

Since we don’t see any errors, let’s fire up the browser and check our page (I have SSH Tunnel enabled, so I can access the webpage with localhost IP and default port(3000):

Our Website is running fine.

How did our website’s content get rendered ? Let us understand the flow of events. First, let me just remind you how the structure of your web application looks like:

Below diagram shows the flow of events from different files when you access your web page:

Whenever you access any web page built in mojolicious, the first file that gets called is the library file: lib/myWebSite.pm. The line $r->get('/')->to('example#welcome'); in the file basically tells to go to the controller Example.pm and execute the welcome sub-routine. Controllers, as you can see in the above screenshot are stored in lib/myWebSite/Controller.

In the controller we see the below lines:

# Render template "example/welcome.html.ep" with message
$self->render(msg => 'Welcome to the Mojolicious real-time web framework!');

This basically tells Mojolicious to set a value to the variable $msg and render the template. But which template ? We don’t see any template name mentioned in the controller, then which page is getting rendered?? Well, since no template is explicitly called Mojolicious will look for the template is the path similar to the controller path: example/welcome. In the above directory structure, we can see a template: template/example/welcome.html.ep, that’s what is getting rendered in the home page now. Ofcourse, you can explicitly mention the template name in the controller and it will exactly behave the same way. Just pass the template name like this:

  $self->render(msg => 'Welcome to the Mojolicious real-time web framework!',template => "example/welcome");

I prefer explicitly mentioning the template names in my controller, to avoid any confusion.

Now that we know that our main page’s content is being displayed by myWebSite/templates/example/welcome.html.ep and the contents looks like this:

% layout 'default';
% title 'Welcome';
<h2><%= $msg %></h2>
<p>
  This page was generated from the template "templates/example/welcome.html.ep"
  and the layout "templates/layouts/default.html.ep",
  <%= link_to 'click here' => url_for %> to reload the page or
  <%= link_to 'here' => '/index.html' %> to move forward to a static page.
</p>

We will change it to add our content:

% layout 'default';
% title 'Home Page';
<h2><%= $msg %></h2>
<p> This is my personal site built with Mojolicious. You can provide testimonials using the link <a href="#">here</a>. 
</p>

If you notice that the heading in the above html content is displayed by the variable $msg. This was set by our controller: myWebSite/lib/myWebSite/Controller/Example.pm. Let us open the controller file and change the below line:

# Render template "example/welcome.html.ep" with message
$self->render(msg => 'Welcome to the Mojolicious real-time web framework!');

to

# Render template "example/welcome.html.ep" with message
$self->render(msg => 'Welcome to My Personal Website!',template => "example/welcome");

Not only did I change the heading, I also explicitly mentioned which template to render.

Lets run our web app with morbo:

morbo myWebSite/script/myWebSite

Refresh our web browser:

Working as expected. Before we wind up, lets just change the default controller and template files to our customized file names:

cp myWebSite/lib/myWebSite/Controller/Example.pm myWebSite/lib/myWebSite/Controller/CustomController.pm

mkdir myWebSite/templates/myTemplates

cp myWebSite/templates/example/welcome.html.ep myWebSite/templates/myTemplates/homepage.html.ep

New Controller Name : myWebSite/lib/myWebSite/Controller/CustomController.pm
New Template Name : myWebSite/templates/myTemplates/homepage.html.ep

Open the new controller(CustomController.pm) and change the first line to reflect the new controller name, from:

package myWebSite::Controller::Example;

to

package myWebSite::Controller::CustomController;

Now change the below line in the same file to point to our newly created template: myTemplates/homepage

# Render template "example/welcome.html.ep" with message
$self->render(msg => 'Welcome to My Personal Website!',template => "example/welcome");

to

# Render template "myTemplates/homepage.html.ep" with message
$self->render(msg => 'Welcome to My Personal Website!',template => "myTemplates/homepage");

Finally, open the main library file: myWebSite/lib/myWebSite.pm and change the below line to call our now custom controller rather than the default:

$r->get('/')->to('example#welcome');

to

$r->get('/')->to('CustomController#welcome');

Lets start our web app and refresh the browser:

morbo myWebSite/script/myWebSite

You will not see any changes since we didn’t change the content. But we also didn’t see any errors that means we have achieved what we want.

We successfully created the home page of your website. In the next session, we will learn about templates and layouts. We will try to add some color to our dull home page that we created in this article.

The post Mojolicious (Part 1): Build Full App appeared first on The Curious Technoid.

I strongly suggest you visit the official website for in depth documentation:

Articles on Mojolicious are published with an assumption that you have basic knowledge on Perl and fundamental idea on how web pages work.

In this article, we will look at Mojolicious::Lite, a Micro real-time web framework(as it’s author puts it). It’s a light weight domain specific language built around Mojolicious. This can be used for your tiny one page projects. We will cover the following topics in this blog:

1. Installation of Mojolicious
2. Building a simple “Hello World” app with Mojolicious::Lite
3. Testing
4. Deployment

Installation

There are 2 different methods to install Mojolicious. You can chose whichever method you find convenient:

Method 1

Installation of the powerful Mojolicious framework can be achieved using only a single-line command. Open the terminal in your server and run the below command:

curl -L https://cpanmin.us | perl - -M https://cpan.metacpan.org -n Mojolicious

^{*source: https://mojolicious.org}

That’s all, this will install the full Mojolicious framework.

Method 2

Alternatively, you can also install it with traditional CPAN command similar to installing any Perl module:

cpanm Mojolicious

OR

perl -MCPAN -e shell
install Mojolicious

This also installs full Mojolicious framework that required to build web applications.

Trust me when I say it, use the Method 1, it’s faster to install that way.

“Hello World” with Mojolicious::Lite

We have Mojolicious installed in our server by following the previous step, now lets go ahead and create a lite_app and try to bring the famous “Hello World” page live in that server.

You can create the lite_app using the below command. lite_app are usually used for simple projects involving probably a single page. Let us create a lite_app called hellomojo using the command:

mojo generate lite_app hellomojo

The above command will create a file called hellomojo in your current directory:

The hellomojo file created is a working sample webpage. You can start building your project on top of this file. The contents of the file will look something like this:

#!/usr/bin/env perl
use Mojolicious::Lite;

get '/' => sub {
  my $c = shift;
  $c->render(template => 'index');
};

app->start;
__DATA__

@@ index.html.ep
% layout 'default';
% title 'Welcome';
<h1>Welcome to the Mojolicious real-time web framework!</h1>

@@ layouts/default.html.ep
<!DOCTYPE html>
<html>
  <head><title><%= title %></title></head>
  <body><%= content %></body>
</html>

The sample file is easily understandable for anyone familiar with Perl. The contents after _DATA_ is treated as templates and layouts that the code uses to display the web content. We will see more about templates and layouts when we build a test project with full_app in future sessions.

For our “Hello World” page, we will change the contents of hellomojo to look like this:

#!/usr/bin/env perl
use Mojolicious::Lite;

get '/' => sub {
  my $c = shift;
  $c->render(template => 'index');
};

app->start;
__DATA__

@@ index.html.ep
% layout 'default';
% title 'Hello World Tutorial with Mojolicious';
<h1>Hello World!!</h1>

@@ layouts/default.html.ep
<!DOCTYPE html>
<html>
  <head><title><%= title %></title></head>
  <body><%= content %></body>
</html>

We only changed the below two lines in the file i.e., the window title and the content.

% title 'Hello World Tutorial with Mojolicious';
<h1>Hello World!!</h1>

Testing

We changed the default sample page’s content to add our “Hello World” text. Now let’s go ahead and test the page. We can test the pages created in Mojolicious using the development web server morbo. Lets not waste any time and go ahead and execute the below command to run our hellomojo web app:

morbo hellomojo

We can see that our page is running. morbo also gives us the URL with port where our website is running:

By default, morbo runs our pages on port 3000. Since morbo is a development web server, debugging is enabled by default and the web server runs in the foreground with debug messages. As we don’t see any errors on the terminal, the page looks fine.

We won’t be able to access the webpage from the local Desktop since port 3000 will not be open in the server. Either we open the port in the firewall, or use SSH Tunneling. I prefer SSH Tunneling, read more about SSH Tunneling here.

Let us go ahead and start a SSH Tunnel to our server from the local Desktop using SSH Keys:

Now that the SSH Tunnel is active, lets go ahead and check our page from our browser using the server’s localhost IP(since SSH Tunnel is running) and morbo’s port(3000): http://127.0.0.1:3000

Success!! We have a working “Hello World” page ready to be deployed.

Deployment

Now that we tested and made sure that our “Hello World” page is working fine, we will deploy it in the server so that it runs all the time. There are different production web servers which gets shipped with Mojolicious (you can read more about it here), I use hypnotoad – ALL GLORY TO THE HYPNOTOAD!. It’s a recommended production grade web server to run Mojolicious web apps.

The command to run your page using hypnotoad is similar to morbo:

hypnotoad hellomojo

hypnotoad by default runs on port 8080(which can be changed ofcourse) and runs in the background. Lets fire up the terminal and establish a new SSH Tunnel to our web server on port 8080 this time. Now we can access our “Hello World” page using the webserver’s local host IP(because we are connected with SSH Tunnel) on port 8080: http://127.0.0.1:8080

There we have it. Our “Hello World” webpage is running to serve the whole world with it’s simplicity and humbleness.

This was a little introduction as to how to build and deploy Mojolicious apps. In the next few sessions, we will build a test project using Mojolicious full_app covering templates, layouts, session management and database connections. Stay tuned.

I strongly suggest you to go through the video tutorials in this website called the MojoCasts. You can find tremendously useful tutorials on Mojolicious in that website. You can also check this informative tutorial written by Ashutosh in dev.to.

The post Mojolicious: “Hello World” with Mojolicious::Lite appeared first on The Curious Technoid.

Syntax

Below is the syntax used to create a SSH Tunnel:

ssh -N -L LOCAL_PORT:HOST:HOST_PORT USER_NAME@REMOTE_SERVER

• -Nrestricts you from inputting any command after the connection is established, this is preferred while using SSH Tunneling
• -L tells SSH that it’s local port forwarding
• LOCAL_PORT The port we will use locally
• HOST The host address in the remote server. Usually localhost / 127.0.0.1 to access the application running in remote server.
• HOST_PORT The port number of the host address where the application is running
• USER_NAME User Name of the remote server
• REMOTE_SERVER Remote Server IP Address or hostname

Use Case: Connecting to remote database

If you host your websites in one of the remote VPS servers, you would have stumbled upon the issue of connecting to the database from your local Desktop. I can access the database locally by opening the DB ports in firewall. But, what if I don’t like opening the DB ports ? The solution: SSH Tunneling. We can create a SSH Tunnel from our local Desktop to the VPS server using SSH. Port forward to the remote DB ports, after which we should be able to connect to the database in the remote server as if they are installed locally.

For the purpose of this demo, I have created a new server in digital ocean and have installed mariaDB database in it.

Setup

Local Desktop:
Operating System: macOS Catalina
Database Client: sequel Pro

Remote Server:
Provider: Digital Ocean
Operating System: Fedora 31 (Cloud Edition)
Database Server: mariaDB

Create SSH Tunnel

First we will create the SSH Tunnel by running the below command in terminal:

ssh -N -L 2000:127.0.0.1:3306 demo@161.35.77.93 -i ~/.ssh/id_curioustechnoid

• 2000 is the local port that we will use in the client
• 127.0.0.1 is the Remote Server localhost. This is nothing but localhost in the remote server
• 3306 is the Remote Server mariaDB Port
• demo is the Remote SSH User Name.
• 161.35.77.93 This is the Remote Server IP Address
• ~/.ssh/id_curioustechnoid is the private SSH Key used to connect to server. Read more about SSH Keys here.

Change the values based on your setup. If all the options are provided correctly we should have an active SSH Tunnel created.

What the above command did is that it created a SSH Tunnel which port forwards any request on port 2000 to the remote server’s port 3306. In our case mariaDB is running in the default port 3306 in the remote server. In the above command, we are using SSH Keys to connect to the server. It’s strongly recommended to use SSH Keys to connect to remote servers instead of username/password. Click here to know more about SSH Keys.

Connect to database

We now have encrypted connection established between local and remote server using the above SSH Tunnel command. Lets try to connect to the database in the remote server.

I will be using sequelPro database client and will connect to root database user. Enter the following details in connection section:

We have successfully connected to the remote database using SSH Tunneling. You would have noticed that we are using localhost(127.0.0.1) and localport(2000) to connect to the database hosted in remote server. The practical usage of this concept in limitless.

Hope this article helps you in using SSH Tunnels. Please feel free to share your experience or your issues in the comment section below. Happy to help!

The post SSH Tunneling (Port Forwarding) – a use case appeared first on The Curious Technoid.

Today we will cover the below topics which will help you start using SSH keys right away:

• Generation of SSH Keys
• Installation of SSH Keys
• Connecting using SSH Keys
• SSH Config File

How to generate SSH Keys

We can generate the SSH Key pair using the ssh-keygen command. You have the option to choose from 4 encryption algorithms that ssh-keygen supports as of this writing. The 4 algorithms are rsa, dsa , ecdsa and ed25519. I always use ecdsa which is a strong algorithm that meets modern security standards. It is highly recommended you generate the SSH key pair using a passphrase. This ensures that, if someone gets their hand on your private key, they wouldn’t be able to use it without the passphrase.

In your local Desktop / Server you can generate SSH Key pair using the below command:

ssh-keygen -t ecdsa

When you the above command is executed, you will be prompted with the below (optional) inputs:

• Enter file in which to save the key (/root/.ssh/id_ecdsa): – If you don’t pass any value to this, the key-pair will be created in the ~/.ssh folder by default, with the name id_ecdsa. You can always enter your preferred name to override this.
• Enter passphrase (empty for no passphrase): – This accept empty sting as input if you don’t wish to use a passphrase. However, I strongly recommend you provide a passphrase so that no-one else will be able to access your private key.

Once the keys are generated, you will find two files created by ssh-keygen, in my case they are: id_curioustechnoid and id_curioustechnoid.pub

id_curioustechnoid : This is your private key
id_curioustechnoid.pub : This is your public key which we need to install in the remote server

How to install SSH Keys

Now that the SSH Keys are generated, let’s go ahead and install them.

1. Install the private key in your local Desktop / Server

Installation of your private key in your local Desktop / Server is pretty straightforward and easy. You just copy your private key to ~/.ssh directory (if your key is not already there):

First create the ~/.ssh directory if it doesn’t exist:

mkdir ~/.ssh
chmod 700 ~/.ssh

Followed by moving the private key to that directory

mv id_curioustechnoid ~/.ssh

That’s it! The private key is now installed in your local Desktop or Server.

You can keep your private key in any directory that you want, it’s a good practice to keep them in ~/.ssh folder in your home directory. If you run the ssh-keygen command using all default values, your public-private key pair will be created in ~/.ssh directory.

2. Install the public key in the remote server

Let’s see how we can install the public key in the remote server to which you wish to connect.

Firstly, transfer the public key that you created to your server using your favourite sftp tool. Since I am connected to a headless demo server without GUI I will use the good old scp command for this purpose. I will be transferring the key to my development.local server to which I will be connecting to.

scp id_curioustechnoid.pub guest@development.local:~/

What this will do is copy the public key that we just created to my development server and place the file in the home directory.

Next we login to the development server using our username/password:

After logging in, append your public key to the SSH authorization file: ~/.ssh/authorized_keys

cat id_curioustechnoid.pub >> ~/.ssh/authorized_keys

Don’t forget to remove the public key file from home directory by running: rm ~/id_curioustechnoid.pub

There you have it, public key successfully installed in the server.

How to connect using SSH Keys

The above steps ensures that the SSH setup is in place, now we can go ahead and connect to our server using the SSH Keys. The below option(-i) tells SSH to use the private key that we just created to connect to development.local server:

ssh guest@development.local -i ~/.ssh/id_curioustechnoid

You will be prompted to enter the passphrase on the screen. Soon after we do, we should be able to successfully connect to the development server.

SSH config file

If you connect to multiple servers like me, managing the keys becomes cumbersome and hard to remember. That’s where SSH config file comes in handy. It’s a master configuration file where you can list all your SSH connections and it’s respective keys. This makes accessing any server more convenient.

Let’s go ahead and create a config file and enter the details of our development server:

1. SSH by default recognizes the config file under ~/.ssh directory.

Let us create the SSH config file using the below command and give it the correct permission(600):

touch ~/.ssh/config
chmod 600 ~/.ssh/config

2. Enter your SSH details in the below format

Open the file ~/.ssh/config using your favorite editor:

Host dev
    HostName development.local
    User guest
    IdentityFile ~/.ssh/id_curioustechnoid


Host other-server
    HostName 192.168.1.232
    User root
    IdentityFile ~/.ssh/id_ecdsa

You can keep adding any number of server details in this file. Once the server details are added, connecting to the server is a breeze:

ssh dev

This will prompt you for the passphrase. You enter the passphrase and you will be logged into the development server. Pretty handy isn’t it.

There you go, we just created, configured and installed SSH keys.

The post How to use SSH Keys appeared first on The Curious Technoid.

Hooks enable us to extend the HR APIs’ capabilities to add additional custom features and validations. Hooks are easier to implement and can be used wherever HR APIs are called, be it a custom interface script or a seeded Self-Service request, if APIs are called then your custom validation will be invoked.

HR APIs are some times life saver, where-in the business wishes to add custom validations on a form or a self service page, rather than enhancing it, you just add your validation and you are good to go.

How to use HR API Hooks?

Tested In: Oracle EBS 12.1.1, 12.1.3, 12.2.6

We will take an example to create user hook on HR_EMPLOYEE_API. The sample business case is, I should not be able to hire anyone who is single.

1 .Get the Module ID and Hook ID

Query to get the API_MODULE_ID of the API we will be using, in my case HR_EMPLOYEE_API:

SELECT *
FROM HR_API_MODULES
WHERE module_package = 'HR_EMPLOYEE_API'
AND module_name = 'CREATE_EMPLOYEE' ;

Get the API_HOOK_ID for the module ID(1195), in my case BP – Before Process

SELECT API_HOOK_ID -- 2690
FROM HR_API_HOOKS
WHERE api_module_id = 1195 --Module ID from the previous query
AND API_HOOK_TYPE = 'BP';

2. Create your custom package

CREATE OR REPLACE PACKAGE XXHR_CUSTOM_HOOKS_PKG
AS
-- Make sure to use only those parameters available in the API are used
PROCEDURE validate_status(p_marital_status in varchar2 default null);
--
END XXHR_CUSTOM_HOOKS_PKG;
/


CREATE OR REPLACE PACKAGE BODY XXHR_CUSTOM_HOOKS_PKG
AS
   PROCEDURE validate_status(p_marital_status in varchar2 default null)
IS
   BEGIN
      --If the marital status is single, throw an error
      IF p_marital_status = 'S'
      THEN
         RAISE_APPLICATION_ERROR(-20006,'Custom Error: Single people cannot join our organization');
      END IF;
   END;
--
END XXHR_CUSTOM_HOOKS_PKG;
/

3. Create User Hook

User hooks can be created using the below script by passing the custom package we created the previous step:

SET SERVEROUTPUT ON;
--
DECLARE
   l_api_hook_call_id        NUMBER;
   l_object_version_number   NUMBER;
   l_sequence                NUMBER;
BEGIN
   SELECT hr_api_hooks_s.NEXTVAL
     INTO l_sequence
     FROM DUAL;

   hr_api_hook_call_api.create_api_hook_call
       (p_validate                   => FALSE,
        p_effective_date             => TO_DATE ('01-JAN-1952', 'DD-MON-YYYY'),
        p_api_hook_id                => 2690,  -- Retrieved Previously
        p_api_hook_call_type         => 'PP',
        p_sequence                   => l_sequence,
        p_enabled_flag               => 'Y',
        p_call_package               => 'XXHR_CUSTOM_HOOKS_PKG',
        p_call_procedure             => UPPER ('validate_status'), -- Custom procedure
        p_api_hook_call_id           => l_api_hook_call_id,
        p_object_version_number      => l_object_version_number
       );
   DBMS_OUTPUT.put_line ('l_api_hook_call_id ' || l_api_hook_call_id);
   COMMIT;
END;
/

4. Register User Hook / Run Pre-Processor

DECLARE
    l_api_module_id number;
BEGIN

    l_api_module_id := 1195; -- Module ID Retrieved from previous query

    --
    -- Create all hook package body source code for one API module
    --
    hr_api_user_hooks_utility.create_hooks_one_module(l_api_module_id);

    --
    -- Build the report text
    --
    hr_api_user_hooks_utility.write_one_errors_report(l_api_module_id);
END;
/

Don’t forget to check the status of the hook call after running the pre-processor by using the below query:

SELECT status, encoded_error
FROM HR_API_HOOK_CALLS
where call_package = 'XXHR_CUSTOM_HOOKS_PKG'
and call_procedure = 'VALIDATE_STATUS';

What does the status mean?

N – New / Not Active
I – Invalid / Errored
V – Valid

If the pre-processor succeeded in registering your Hook call, then the status will be V if not then the status will be I. There is no direct way of finding the reason for the pre-processor to have failed. There is a script which can help you to find the error details which we will look into now. Run the below scripts in the same session where you ran pre-processor script to find the errors if any:

Note: This will only work if you have ran the hr_api_user_hooks_utility.write_one_errors_report(l_api_module_id); package call when you ran pre-processor as seen in the pre-processor script above.

/* Run the below script to get the error output */
set heading off
set feedback off
select text
from hr_api_user_hook_reports
where session_id = userenv('SESSIONID')
order by line;

/* Don't forget to clear the data from report tables by running the below script */
execute hr_api_user_hooks_utility.clear_hook_report;

5. Delete User Hook

If you wish to delete the user hook, you can use the below script by passing api_hook_call_id and object_version_number:

SET SERVEROUTPUT ON;
DECLARE
    --
    ln_object_version_number NUMBER;
    --
BEGIN
    /* You can get the api_hook_call_id by using the below query:
    SELECT api_hook_call_id, object_version_number
    FROM HR_API_HOOK_CALLS
    where call_package = 'XXHR_CUSTOM_HOOKS_PKG'
    and call_procedure = 'VALIDATE_STATUS';
    */
    hr_api_hook_call_api.delete_api_hook_call (p_api_hook_call_id => 1112,
                                               p_object_version_number => 1);
    DBMS_OUTPUT.put_line ('DELETED HOOK...');
EXCEPTION
    WHEN OTHERS
    THEN
       DBMS_OUTPUT.put_line (SUBSTR ('Error: ' || SQLERRM, 1, 255));
END;
/

Important tables of Hooks:

SELECT * FROM HR_API_MODULES -- All Hook Modules can be found here

SELECT * FROM HR_API_HOOKS -- All hooks can be found here

SELECT * FROM HR_API_HOOK_CALLS -- All custom hook package calls can be found here

Caution:
1. API Hooks gets invoked if and only if HR API’s are called.
2. In quite a few cases HR API Hooks cannot be used where APIs aren’t called, few examples are:

• • Certain forms like People Screen does not call API hence hooks will not work. (That’s where custom.pll or forms personalization comes into play)
  • If you have configured a Special Information Type(SIT) with approvals, in Employee Self Service for employee’s to request for something, then don’t use hooks as API’s gets invoked only at the time of approval

The post HR API Hooks / User Hooks appeared first on The Curious Technoid.

If you are not aware, fail2ban is a brute-force attack mitigation tool. This post will guide you to quickly install fail2ban with basic configuration. In this article I will be showing how to configure fail2ban for SSH brute-force attacks.

Let’s cut to the chase and start installing fail2ban.

For the purpose of this demonstration I shall be using Fedora 31. You should be able to find similar commands in your respective distros.

Installation

Installation is pretty straight forward similar to installing any other software in Linux. I will use the DNF package manager which is the default for Fedora now.

dnf install fail2ban -y

Configuration

In fail2ban, setup for a particular service like SSH is called as jail. Like any intruder being put into a jail, any intruder attempting to gain access to your server via SSH would be put inside “SSHD jail” and anyone trying to gain access via SFTP would be put inside “SFTP jail”. Controlling the parameters to be “eligible” to be put into these jails are handled by jail configuration file.

There are 2 configuration files in fail2ban:

• /etc/fail2ban/jail.conf
• /etc/fail2ban/fail2ban.conf

It is recommended not to edit these files, but to create a copy of them with ‘.local’ file extension. Once you have created the copy change the desired values in those files.

fail2ban.conf file has the configuration to control fail2ban’s behaviour. For instance you can set the log level parameters here.
jail.conf file is where you enable services (like ssh, sftp, apache) which you wish to monitor for intrusion. You also define criteria’s on when to block the IPs.

Below are the brief steps I usually follow to setup fail2ban to mitigate SSH brute-force attacks on my servers.

1. Create ‘local’ copies of config files:

 cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
 cp /etc/fail2ban/fail2ban.conf /etc/fail2ban/fail2ban.local

2. Enable SSHD jail in jail.local file

First run the below command to check the active jails configured(should be none obviously):

fail2ban-client status

Since this is a fresh installation we can see that no jail has been configured.

Open the jail.conf using your favorite editor(obviously it’s vim!). Find the section where it says [sshd]. You might find 2 entries, 1 commented and the other uncommented, go to the uncommented entry which should look like this:

Change it to make it look something like this:

#
# SSH servers
#

[sshd]

# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
#mode   = normal
enabled = true
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
maxretry = 3

What did we exactly do here ?

• enabled : This enables the SSHD jail for fail2ban to keep monitoring the SSH connections to your server.
• maxretry : This defines how many incorrect password tries will you allow users trying to connect to your server via SSH. Beyond this, fail2ban will block them.

3. Additional Configuration

You can skip to step 4, if you want to ignore the below additional configurations but trust me you would want to know about these steps

I prefer to change few other parameters in jail.local when I setup fail2ban.

Ignoring IPs from being blocked

I have this habit of adding my own IPs to exception list so that I never get blocked by fail2ban. You might want to add your work IPs to this list so that you don’t get blocked while connecting from office in case you enter incorrect credentials multiple times. To add the the IPs that you want to exclude from monitoring, add them to the ignoreip parameter.

Uncomment and change the below line in jail.local

#ignoreip = 127.0.0.1/8 ::1

to

ignoreip = 127.0.0.1/8 ::1 192.168.1.119

Assuming that my local IP that I want to exclude from blocking is 192.168.1.119, fail2ban will not “ban” this IP no matter how many times I enter incorrect credentials. You can add any number of IPs separated by space or comma.

Changing bantime and findtime

Fail2ban gives you the ability to customize certain things like When to Ban? How long to Ban?

findtime parameter instructs fail2ban, within what duration should the “maxretry” value be checked. We had set the maxretry value(in sshd setup above) to 3, lets say we set findtime value to 60mins. So within 60mins if there are 3 incorrect attempts to login, then source IP gets added to ban list.

bantime parameter defines how long the IP should be banned, you may wish to ban the IP for a day, a week or a month. This parameter instructs fail2ban the same thing.

For example, lets say our maxretry value is 3 in SSHD configuration. You want to ban users if he enters incorrect password 3 times within a span of 10mins and block him from further attempting to try logging in for 3 days. You will change the value as below:

findtime = 10m

bantime = 76h

Enable logging via systemd journals

Fedora has a problem where unless logging is directed via systemd journald, it throws an error while starting fail2ban. To avoid this issue, you need to tell fail2ban to log via systemd journald by changing the below parameter:

backend = auto

to

backend = systemd

Tell fail2ban to use firewalld to block the IPs

I use firewalld as my firewall management tool so I tell fail2ban to use it to block the IPs by changing the below parameter

# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
banaction = iptables-multiport
banaction_allports = iptables-allports

changed to:

# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
banaction = firewallcmd-ipset
banaction_allports = firewallcmd-ipset

Verify and start fail2ban

Once we have all the changes done, we can verify our setup by running the command:

fail2ban-client -x start

fail2ban will tell us that set up in looking good.

Next step is to enable fail2ban so that it starts at boot time automatically. Then we proceed to start the fail2ban service to start monitoring all SSH connections.

Enable fail2ban so that it automatically starts at boot time

systemctl enable fail2ban.service

Start fail2ban service using the command:

systemctl start fail2ban.service

Make sure fail2ban is running fine

systemctl status fail2ban.service

Congratulations! We just enabled SSH jail to start monitoring all SSH connections to your server. You can check your ssh jail status by running the below command:

fail2ban-client status

You can check all the blocked IPs by running the command:

fail2ban-client status sshd

You can find the jail.local file with all the configuration discussed in this article here.

Some useful commands to keep in mind

• Check all services monitored by fail2ban: fail2ban-client status
• Check the blocked IP List by ssh jail: fail2ban-client status sshd
• Remove blocked IPs: fail2ban-client set sshd unbanip 192.168.143.191
• Remove all blocked IPs: fail2ban-client unban --all

Enable Debugging

In case you stumble upon any issues, we can enable debugging to get more information.

1. Open the file: /etc/fail2ban/fail2ban.local and change the below parameter

loglevel = INFO

Change to

loglevel = DEBUG

2. Restart the service

systemctl restart fail2ban

3. Monitor the debug messages by looking at the log file which be found at: /var/log/fail2ban.log

tail -f /var/log/fail2ban.log

If you wish to see fail2ban in action, I suggest you head to the video tutorial that I created regarding fail2ban setup. You can drag the video to go directly to 06:32 to see fail2ban in action skipping all the setup which we already discussed in this article.

The post Secure your computer using fail2ban appeared first on The Curious Technoid.

Today we will look into queries to get the number of active users connected to the application and all the active users connect to the (EBS)application.

Tested In: Oracle EBS 12.1.1, 12.1.3, 12.2.6

First lets get the all the active users connected to the application using the below query:

SELECT fnd.node_name
      ,(SELECT user_name 
          FROM fnd_user f 
         WHERE f.user_id = icx.user_id) user_name
  FROM icx_sessions icx
      ,fnd_nodes fnd
 WHERE icx.disabled_flag  <> 'Y'
   AND icx.PSEUDO_FLAG     = 'N'
   AND icx.node_id         = fnd.node_id
   AND (icx.last_connect +
       decode(FND_PROFILE.VALUE('ICX_SESSION_TIMEOUT'),
               NULL,
               icx.limit_time,
               0,
               icx.limit_time,
               FND_PROFILE.VALUE('ICX_SESSION_TIMEOUT') / 60) / 24) > SYSDATE       
   AND counter              < icx.limit_connects 

This query below, will help you to get the number of all active users connected to the application:

SELECT icx.node_id,
       fnd.node_name,
       count(distinct icx.session_id) "ACTIVE_USER_SESSIONS_COUNT"
  FROM icx_sessions icx
      ,fnd_nodes fnd
 WHERE icx.disabled_flag != 'Y'
   AND icx.PSEUDO_FLAG    = 'N'
   AND icx.node_id        = fnd.node_id
   AND (icx.last_connect +
       decode(FND_PROFILE.VALUE('ICX_SESSION_TIMEOUT'),
               NULL,
               icx.limit_time,
               0,
               icx.limit_time,
               FND_PROFILE.VALUE('ICX_SESSION_TIMEOUT') / 60) / 24) >  SYSDATE
   AND counter             < icx.limit_connects
 GROUP by icx.node_id, fnd.node_name 

Hope this helps.

The post Query to check active users in EBS appeared first on The Curious Technoid.

More often than we realize, we see these memes teasing how difficult it is to quit vim if you are new to vim. What better way to start writing about vim than to show “How to Quit Vim”.

Quitting is nothing but closing the open (and hidden) files in vim. There are multiple ways to quit vim:

1. :q

If only 1 file is open and there are no changes to the file, you can quit vim using the below command:

<ESC>:q

If there are any unsaved changes, vim will prompt you saying it cannot exit. Suppose if there are multiple files open in different tabs then it will close the file in the current tab.

2. :q!

If the above command failed to quit vim, you force vim to quit without saving using the below command:

<ESC>:q!

The exclamatory symbol force quits vim without saving. As before, if multiple files are open then it will close the file in the current tab.

3. :qa[ll]

What if there are multiple files that are open and we need to close all the files and quit vim? One way is you keep on hitting :q until all the open files are closed or, alternatively use the below command to quit all the files in one go:

<ESC>:qall

You can also use the shorter version:

<ESC>:qa

But be aware, similar to :q this will close all the files only if there are no unsaved files. If there are then vim will prompt you that it cannot quit.

4. :qa[ll]!

I guess by now this is pretty self explanatory, it will close all the files before quitting vim even if there are unsaved changes. The unsaved changes will be lost.

5. :wq <or> : x

If you wish to save and close the file then use the below command:

<ESC>:wq

You can also use the single letter command:

<ESC>:x

This will save the open file in the current tab and close that file. This will quit vim only if there are only 1 tab that are open. If there are multiple files open in different tabs, then vim will close the current file and bring the file in the next tab ready to be edited.

6. :wqa[ll]

This is pretty self explanatory , it will save all the files and close all the files before quitting vim. Unlike :wq which might or might not quit vim, this will quit vim after taking care of all the files.

Hope after reading this article you will never be stuck inside vim again.

The post How to Quit Vim ? appeared first on The Curious Technoid.

FNDLOAD command is run from the application server. This command needs database apps password for uploading and downloading AOL objects. FNDLOAD command can very well be used in shell scripts to automate the migration activity.

FNDLOAD is present in $FND_TOP/bin folder. If your $FND_TOP folder is set then you can directly use FNDLOAD instead of the complete path.

The structure of FNDLOAD goes like this:

$FND_TOP/bin/FNDLOAD <db_user_name>/<db_password> 0 Y <UploadDownloadMode> <AOLSpecificScriptName> <FileWhereYourSetupWillBeStored> <EntityName> <AOLSpecificParameters>

You can use the wild character ‘%’ to extract multiple AOL objects. For any specific parameters, you can check the <AOLSpecificScriptName>, the variables starting with a colon(:) can be used as parameter in the FNDLOAD commands.

Below are few of the commonly used FNDLOAD sample script.

Tested In: Oracle EBS 12.1.1, 12.1.3, 12.2.6

Lookups

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y DOWNLOAD $FND_TOP/patch/115/import/aflvmlu.lct XXHR_CUSTOM_LOOKUP.ldt FND_LOOKUP_TYPE APPLICATION_SHORT_NAME="XXHR" LOOKUP_TYPE="XXHR_CUSTOM_LOOKUP"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/aflvmlu.lct XXHR_CUSTOM_LOOKUP.ldt UPLOAD_MODE=REPLACE CUSTOM_MODE=FORCE

Messages

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y DOWNLOAD $FND_TOP/patch/115/import/afmdmsg.lct XXHR_CUSTOM_MESSAGE.ldt FND_NEW_MESSAGES APPLICATION_SHORT_NAME="XXHR" MESSAGE_NAME="XXHR_CUSTOM_MESSAGE"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/afmdmsg.lct XXHR_CUSTOM_MESSAGE.ldt UPLOAD_MODE=REPLACE CUSTOM_MODE=FORCE

Concurrent Programs

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y DOWNLOAD $FND_TOP/patch/115/import/afcpprog.lct XXHR_CONCURRENT_PROGRAM.ldt PROGRAM APPLICATION_SHORT_NAME="XXHR" CONCURRENT_PROGRAM_NAME="XXHR_CONC_SHORT_NAME"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/afcpprog.lct XXHR_CONCURRENT_PROGRAM.ldt UPLOAD_MODE=REPLACE CUSTOM_MODE=FORCE

Form Functions

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y DOWNLOAD $FND_TOP/patch/115/import/afsload.lct XXHR_CUSTOM_FUNC.ldt FUNCTION FUNC_APP_SHORT_NAME='XXHR' FUNCTION_NAME='XXHR_CUSTOM_FUNC'

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y UPLOAD $FND_TOP/patch/115/import/afsload.lct XXHR_CUSTOM_FUNC.ldt

Menus

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y DOWNLOAD $FND_TOP/patch/115/import/afsload.lct XXHR_CUSTOM_MENU.ldt MENU MENU_NAME="XXHR_CUSTOM_MENU"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/afsload.lct XXHR_CUSTOM_MENU.ldt

Responsibilities

Download:

FNDLOAD apps/<apps_password> O Y DOWNLOAD $FND_TOP/patch/115/import/afscursp.lct XXHR_CUSTOM_RESP.ldt FND_RESPONSIBILITY RESP_KEY="XXHR_RESP_KEY"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/afscursp.lct XXHR_CUSTOM_RESP.ldt

XML Publisher Data Definitions

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y DOWNLOAD $XDO_TOP/patch/115/import/xdotmpl.lct XXHR_CUSTOM_REPORT_DATA_DEFINITION.ldt XDO_DS_DEFINITIONS APPLICATION_SHORT_NAME=XXHR DATA_SOURCE_CODE=XXHR_CUSTOM_REPORT_DATA_DEFINITION

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> O Y UPLOAD $FND_TOP/patch/115/import/afsload.lct XXHR_CUSTOM_REPORT_DATA_DEFINITION.ldt

Alerts

Download:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y DOWNLOAD $ALR_TOP/patch/115/import/alr.lct XXHR_CUSTOM_ALERT.ldt ALR_ALERTS APPLICATION_SHORT_NAME=XXHR ALERT_NAME="Custom Alert"

Upload:

$FND_TOP/bin/FNDLOAD apps/<apps_password> 0 Y UPLOAD $ALR_TOP/patch/115/import/alr.lct XXHR_CUSTOM_ALERT.ldt CUSTOM_MODE=FORCE

You can find more information in the documentation provided by oracle: Oracle Documentation

Happy Migrating.

The post FNDLOAD – A Boon to EBS Developers appeared first on The Curious Technoid.