I want to install a custom keyboard layout on my Lubuntu 14.04
I added my layout in /usr/share/X11/xkb/symbols/<language file>

and I added the appropriate configurations in base.xml, base.lst, evdev.xml and evdev.lst under /usr/share/X11/xkb/rules but the IBus Preferences applet is not showing the new layout under the language I added it under.

How can I enable the new layout so I can use it?

Have you tried
That's how I set my map for dvorak in Fluxbox. I have never tried a completely from-scratch map, though.

That sets the language but not the layout. I tried all the different options for setxkbmap and I was still not able to load my keyboard layout.

Firstly, you do need to tell us your distro and desktop environment when you ask a question, because the answers may vary.

Most keyboard configuration tools do not look to see what drivers are in the directory, but rely on a built-in list. If you are just going to re-write the basic driver, keep the name the same. My driver "gb" is still qwerty, but has 14 other keys changed. You will only need a different name if you are creating two versions. So my "gb" has phonetic symbols on the third level, but I have another driver with astrological ones: "☉" instead of "ə" on "a". That I call "astro", so I need a way of entering it. The solutions are

1. If your desktop has a configuration editor (sometimes not installed by default), you will find a place under keyboard (which may be under peripherals) where the drivers are listed, like
layouts [gb,gr, astro]
and you can just type in the new name.

2. Otherwise, you need to search the home directory for the desktop's configuration files, and edit the keyboard one.

Lubuntu comes with LXDE.

I thought the keyboard config tool ("IBus Preferences" in Lubuntu) looks at the config files under X11? Are you saying that's not the case?
Is there another way to switch between keyboard layout?

Sorry I missed the Lubuntu bit: I just looked at the profile. As I said, the list is built-in. If it weren't, the program would give you a useless list of options — ie, il, in — that no-one would understand. If you use my second method, it will work.