Category Archives: Software

Getting current memory usage

I’ve started gathering the information that I need in order to be able to create the user interface that I want for my PI3 home server.

One of the things I want to do is write a program that will run all the time that the pi is online, it’ll be started and stopped by systemctl just like open vpn etc.

What this program will do is one a second take some vital statistics of my PI and upload it to a MySQL database.   The first part of the information that I wanted to gather is the memory usage of my machine.  In order to be able to properly calculate the ram usage, I needed to get all the different types of memory allocation.  The reason for this is the “used” memory in linux simply means memory that is being used for something, this does not mean that it’s not available.   On the linux OS  you can execute the command “free” to get the amount of memory used

 

free
             total       used       free     shared    buffers     cached
Mem:        234452     217780      16672      10564      36684     100736
-/+ buffers/cache:      80360     154092
Swap:       102396         20     102376

As you can see from the output above. you get lots of different pieces of information.  To find out the true amount of used ram, you can user this formula

 

actual used = total – (free + shared + buffers + cached)

So using the figures above.

actual used = 234452 – ( 16672 + 10564 + 36684 + 100736 )

actual used = 234452 – 164656

actual used = 69796

That’s alot less than the reported 217780 of used Memory.   The reason for this is that shared, buffers and cached memory can be dumped by the operating system to make way for other processes.  So, on my PI, (It’s a Model B 256MB ram) it that will happen sooner than on a PI2 or PI3, or even a later PI.

 

There are several ways to get at the system memory using raspbian.

 

cat /proc/meminfo
MemTotal: 234452 kB
MemFree: 16124 kB
MemAvailable: 128144 kB
Buffers: 37116 kB
Cached: 100736 kB

Shmem:             10564 kB

SwapTotal: 102396 kB
SwapFree: 102376 kB

 

It’s all there at loads more too.  however, it’s shown in kB  which means that there’s information missing, it’s rounded to the nearest kB, and being pedantic I want the exact number of bytes.

 

What about the c command sysinfo?

 

This link provides the details about sysinfo.  Upon reading, it looks promising, however there’s a fatal floor in this plan.  The cached memory isn’t reported, which kinda messes up the plan completely.

 

This really leaves me with one option that I can see so far.  The linux command “free -b”  the -b means return the values as bytes, which is perfect, and the output is similar to above.

 

free -b
             total       used       free     shared    buffers     cached
Mem:     240078848  223744000   16334848   10817536   38113280  103157760
-/+ buffers/cache:   82472960  157605888
Swap:    104853504      20480  104833024

That's the figures that I'm after, just need to get at the them now.

So, to do that here's a little c method that I wrote.

struct MemoryUsage{
 unsigned long RamTotal;
 unsigned long RamActualUsed;
 unsigned long RamUsed;
 unsigned long RamFree;
 unsigned long RamShared;
 unsigned long RamBuffers;
 unsigned long RamCached;
 unsigned long SwapTotal;
 unsigned long SwapFree;
 unsigned long SwapUsed;
};

struct MemoryUsage GetMemoryUsage()
{
 struct MemoryUsage memoryUsage;
 
 memoryUsage.RamTotal = 0;
 
 FILE *fp;
 
 fp = popen("free -b","r");
 if (fp == NULL)
 {
 perror("Error opening file");
 exit(EXIT_FAILURE);
 }
 
 char str[1024];
 int lineNo = 0;
 char token[1024];
 
 while (fgets(str, sizeof(str)-1, fp) != NULL) {
 lineNo++;
 
 if (lineNo == 2)
 {
 sscanf( str, "%s %u %u %u %u %u %u",
 token,
 &memoryUsage.RamTotal,
 &memoryUsage.RamUsed,
 &memoryUsage.RamFree,
 &memoryUsage.RamShared,
 &memoryUsage.RamBuffers,
 &memoryUsage.RamCached
 );
 }
 if (lineNo == 4)
 {
 sscanf( str, "%s %u %u %u",
 token,
 &memoryUsage.SwapTotal,
 &memoryUsage.SwapUsed,
 &memoryUsage.SwapFree
 );
 }
 }
 
 fclose(fp);

 memoryUsage.RamActualUsed = memoryUsage.RamTotal - ( memoryUsage.RamFree + memoryUsage.RamShared + memoryUsage.RamCached );
 
 return memoryUsage;
}


To call it, all you need to do is...

struct MemoryUsage memoryUsage;
 memoryUsage = GetMemoryUsage();

And at this point, you should have all the memoryinformation that you could want.  Including the actual used ram from the formula about.

Building a Lamp box using debian linux

LAMP – Linux, Apache, MySql & PHP.  These software packages installed together make for an extremely powerful way of hosting websites. Various forms of Linux can be downloaded for free.  Apache is the most used web server in the world and it’s free. MySql is a powerful relational database and PHP is a powerful scripting language.  Together these programs provide everything you need to create full websites with ineractiviy.  What’s more with the power of a relational database, you can provide very data rich pages to your users.

Over the next few posts, I’m going to detail a method of setting up a fully working LAMP system.  The steps will allow you to build a new lamp system from scratch to run on either a 64-bit intel based processor, or a Raspberry PI.  Apart from the initial steps for installing the Linux OS, everything else will be pretty much the same.

I’m going to install three different systems with this setup.  The first two will be virtual machines running in VMWare (One on VMWare Player in windows, one on VMWare Fusion on Mac OSX). The third will be installed as the main OS for a Raspberry PI Model B.  As these systems are based on different hardware, I need to use two different installers.  For the virtual machines, I can use the installer for Debian linux from here, for my download I chose the “amd64” download as this is for 64-bit processors (which both my desktop and laptop have)  for the Raspberry Pi I used the Raspbian download from here, it’s worth me pointing out that this will also install Debian Wheezy on the PI.

The installers are all fairly straight forward.  For the Intel based images, you are provided with an ISO file which can be plugged directly into VMWare whilst creating the virtual machine.  For the Raspberry PI, the installation process means using another machine to format and install the image onto the SD Card, then boot the PI.

Minimum install

Installing Debian Wheezy as a virtual macihine

LinuxAfterFirstBoot

Either way, you will end up with the same boot loader and installation options.  For my install, I chose to install  the absolute minimum, no GUI, no tools, or anything, I wanted complete control over the setup process.

Once installed I will be manually installing Apache, PHP, MySQL and an FTP server. This will give me the ability to use the linux machine in the same way that I would use for web hosting.  If I wanted, instead of the FTP, I could use SAMBA to create windows file shares which might work well as an alternative to FTP.  That is a decision that I can make later.  If I desired there is nothing stopping me from having both FTP and SAMBA.

The observant will notice that I after install I executed the APT-GET UPDATE command. This makes sure that my Linux install is completely patched.  I believe that patching is always the better option, it helps with issues like the recent Heartbleed scare.

 

Next post installing MySql

Nested Templates

I’m still pulling content across from my current website into the new layout.  After messing about with dreamweaver for most of the weekend, I created a base template that the entire site will be built upon. (see my post New face of Astronomy.cjdawson.com) However, some of the pages have multiple items in each category, so I wanted to have a menu on the left side of the screen for this.  There are several ways to accomplish this, using PHP is one really flexible method, but for now I decided to use nested templates in dreamweaver.   This technique allows me to create a template based on another template.  So, I can use my base template, as a file which is a template for the projects section.  The project section can have it’s own template which is used for the articles.

This method become really cool when I need to make an adjustment to the base template – for example when I decided to alter the way that the section buttons appear.  I simple alter the base template and it will update all the documents for the entire site.   Everything gets processed correctly, and the change is cascaded throughout the entire site with no effort on my part.

The other way to accomplish the same thing is to have the site programmed up using PHP,  I’m sure that eventually I’ll move over to a PHP data driven web layout but for now, having everything processed locally is perfect for me.

New face of Astronomy.cjdawson.com

Having spend a little time looking around the web, checking out sites, seeing suggestions of layouts and generally having a good look around, I realised that my Astronomy website needs a couple of approaches to solving the display problems.

The website is made complex because there is the need for a both general categories and within that there are for some of the items sub menus needed.  Some of these in time will become quite long – 110 items needed for the messier object list alone.

To add to the fun, the landing page for the site doesn’t need an sub menu items at all. So having an empty menu will look silly. Neither will the links page.

So, how to solve that problem.

New portal page

For a long time I have wanted a good looking portal page.  At first, I wanted to do something “cool”.  I was going to make a collage picture, and have parts of the site linked off that.  For my BTeC I actually started to put it together.  I was good for the course, to prove that I was able to do it and the instructor loved that it was different.  However, I wasn’t happy with it, The images did not lend themselves to a good website design, and it wasn’t intuitive.

Website redesign

astronomy.cjdawson.com screenshot

astronomy.cjdawson.com screenshot

Time has finally come.  After having had my astronomy site up and running for over 10 years, I figured that it is finally time to give my astronomy website a complete make over.  The current site is rather inconsistent and has a very distinct 90’s style.

My plan is to bring the site up to day, give it a much more professional look and feel, and give me a better looking site to update everything from.

Paypal Donate button

After receiving this comment.  I initially posted back saying that I wasn’t interested in making any money from my site.  But then, over the last two weeks I started to think, well if someone really wishes to throw money at me, why should I stop them?   Before I go on, I’d like to state that I am not interested in making money from the site.  I have no interest in turning this into a business.    I decided to do this because I like doing things that I have not done before.

Reinstalling Lion

mac restoring time capsule data

mac restoring time capsule data

Reinstalling Lion on my Mid-2009 MacBook Pro. This is not as straight forward as if you buy a Mac today, but it’s actually not that bad – well, so far.

Sessions

Sessions are an essential part of any interactive website.  A session is what a programmer calls the time whilst the user is making use of the software.  In the case of a website, that is the time from when they first browse to the side, to when they close the webpage.   Each time the user comes to the website, it is considered to be a separate session.