Installshield custom dialog, custom actions, and code

Hi, here I have to share you an interesting post for including a file browse dialog into our Installscript project.

Here we use a Windows API GetOpenFileNameA for the purpose.

We will use two files named brwsdlg.h and Brwsdlg.rul. Follow these simple steps as explained below:

Open the Installation Designer view.
Under Behavior and Logic, select InstallScript.
Right-click on your setup.rul file and select New Script File.
Name it as brwsdlg.h. Similarly create brwsdlg.rul file.
Paste the below code in brwsdlg.h script .

// Avoid multiple include collisions.
#ifndef _BRWSDLG_H_
#define _BRWSDLG_H_

// Options for Flags member of OPENFILENAME.
#define OFN_READONLY 0x00000001
#define OFN_OVERWRITEPROMPT 0x00000002
#define OFN_HIDEREADONLY 0x00000004
#define OFN_NOCHANGEDIR 0x00000008
#define OFN_SHOWHELP 0x00000010
#define OFN_ENABLEHOOK 0x00000020
#define OFN_ENABLETEMPLATE 0x00000040
#define OFN_ENABLETEMPLATEHANDLE 0x00000080
#define OFN_NOVALIDATE 0x00000100
#define OFN_ALLOWMULTISELECT 0x00000200
#define OFN_EXTENSIONDIFFERENT 0x00000400
#define OFN_PATHMUSTEXIST 0x00000800
#define OFN_FILEMUSTEXIST 0x00001000
#define OFN_CREATEPROMPT 0x00002000
#define OFN_SHAREAWARE 0x00004000
#define OFN_NOREADONLYRETURN 0x00008000
#define OFN_NOTESTFILECREATE 0x00010000
#define OFN_NONETWORKBUTTON 0x00020000
#define OFN_NOLONGNAMES 0x00040000
#define OFN_EXPLORER 0x00080000
#define OFN_NODEREFERENCELINKS 0x00100000
#define OFN_LONGNAMES 0x00200000

// CommDlgExtendedError-related defines.
#define CDERR_DIALOGFAILURE 0xFFFF
#define CDERR_DIALOGFAILURE_MSG "The file browse dialog box could not be created."
// The dialog box could not be created. The common dialog box function
// call to the DialogBox function failed. For example, this error occurs
// if the common dialog box call specifies an invalid window handle."
#define CDERR_FINDRESFAILURE 0x0006
#define CDERR_FINDRESFAILURE_MSG "The file browse dialog box function failed to find a specified resource."
// The common dialog box function failed to find a specified resource.
#define CDERR_INITIALIZATION 0x0002
#define CDERR_INITIALIZATION_MSG "The file browse dialog box function failed during initialization."
// The common dialog box function failed during initialization. This error
// often occurs when sufficient memory is not available.
#define CDERR_LOADRESFAILURE 0x0007
#define CDERR_LOADRESFAILURE_MSG "The file browse dialog box function failed to load a specified resource."
// The common dialog box function failed to load a specified resource.
#define CDERR_LOADSTRFAILURE 0x0005
#define CDERR_LOADSTRFAILURE_MSG "The file browse dialog box function failed to load a specified string."
// The common dialog box function failed to load a specified string.
#define CDERR_LOCKRESFAILURE 0x0008
#define CDERR_LOCKRESFAILURE_MSG "The file browse dialog box function failed to lock a specified resource."
// The common dialog box function failed to lock a specified resource.
#define CDERR_MEMALLOCFAILURE 0x0009
#define CDERR_MEMALLOCFAILURE_MSG "The common dialog box function was unable to allocate memory for internal structures."
// The common dialog box function was unable to allocate memory
// for internal structures.
#define CDERR_MEMLOCKFAILURE 0x000A
#define CDERR_MEMLOCKFAILURE_MSG "The common dialog box function was unable to lock the memory associated with a handle."
// The common dialog box function was unable to lock the memory associated
// with a handle.
#define CDERR_NOHINSTANCE 0x0004
#define CDERR_NOHINSTANCE_MSG "The ENABLETEMPLATE flag was set, but you failed to provide a corresponding instance handle."
// The ENABLETEMPLATE flag was set in the Flags member of the initialization
// structure for the corresponding common dialog box, but you failed to provide
// a corresponding instance handle.
#define CDERR_NOHOOK 0x000B
#define CDERR_NOHOOK_MSG "The ENABLEHOOK flag was set, but you failed to provide a pointer to a corresponding hook procedure."
// The ENABLEHOOK flag was set in the Flags member of the initialization
// structure for the corresponding common dialog box, but you failed to provide
// a pointer to a corresponding hook procedure.
#define CDERR_NOTEMPLATE 0x0003
#define CDERR_NOTEMPLATE_MSG "The ENABLETEMPLATE flag was set, but you failed to provide a corresponding template."
// The ENABLETEMPLATE flag was set in the Flags member of the initialization
// structure for the corresponding common dialog box, but you failed to provide
// a corresponding template.
#define CDERR_REGISTERMSGFAIL 0x000C
#define CDERR_REGISTERMSGFAIL_MSG "The RegisterWindowMessage function returned an error code when it was called by the file browse dialog box function. "
// The RegisterWindowMessage function returned an error code when it was called
// by the common dialog box function.
#define CDERR_STRUCTSIZE 0x0001
#define CDERR_STRUCTSIZE_MSG "The lStructSize member of the initialization structure for the corresponding file browse dialog box is invalid. "
// The lStructSize member of the initialization structure for the corresponding
// common dialog box is invalid.
#define FNERR_BUFFERTOOSMALL 0x3003
#define FNERR_BUFFERTOOSMALL_MSG "The buffer pointed to by the lpstrFile member of the OPENFILENAME structure is too small for the filename specified by the user."
// The buffer pointed to by the lpstrFile member of the OPENFILENAME structure
// is too small for the filename specified by the user. The first two bytes
// of the lpstrFile buffer contain an integer value specifying the size, in bytes
// (ANSI version) or 16-bit characters (Unicode version), required to receive the full name.
#define FNERR_INVALIDFILENAME 0x3002
#define FNERR_INVALIDFILENAME_MSG "A filename is invalid."
// A filename is invalid.
#define FNERR_SUBCLASSFAILURE 0x3001
#define FNERR_SUBCLASSFAILURE_MSG "An attempt to subclass a list box failed because sufficient memory was not available."
// An attempt to subclass a list box failed because sufficient memory was not available.

// OPENFILENAME structure. Notice that all string members below are declared
// as LONG. For example, lpstrFilter. Do not use STRING.
typedef OPENFILENAME
begin
LONG lStructSize;
HWND hwndOwner;
HWND hInstance;
POINTER lpstrFilter;
POINTER lpstrCustomFilter;
LONG nMaxCustFilter;
LONG nFilterIndex;
POINTER lpstrFile;
LONG nMaxFile;
POINTER lpstrFileTitle;
LONG nMaxFileTitle;
POINTER lpstrInitialDir;
POINTER lpstrTitle;
LONG Flags;
SHORT nFileOffset;
SHORT nFileExtension;
POINTER lpstrDefExt;
POINTER lCustData;
POINTER lpfnHook;
POINTER lpTemplateName;
end;

// Windows API declares.
prototype comdlg32.GetOpenFileNameA( LONG );
prototype comdlg32.CommDlgExtendedError();
prototype user32.wsprintf(STRING, STRING, POINTER);
// Our file browse API, defined in matching .rul file.
prototype FileBrowseDlg( BYREF STRING, NUMBER, STRING, STRING, STRING, BOOL, LIST , BOOL );

#endif
Paste below code in Brwsdlg.rul file.

#ifndef _BRWSDLG_RUL_
#define _BRWSDLG_RUL_

prototype Kernel32.RtlMoveMemory(BYREF STRING, POINTER, NUMBER);

typedef STR260
begin
STRING sz[260];
end;

///////////////////////////////////////////////////////////////////////////////
// FileBrowseDlg() uses the Windows GetFileNameA function to allow single file
// selection. Callers specify filter, dialog title, and initial browse directory.
//
// Input:
//
// szFile: String variable into which FileBrowseDlg() will place the selected
// file's fully qualified pathname. The variable passed in as szFile must
// be explicitly sized (declared as szMyFile[260] for example) and the
// size must be passed in as the second parameter (nszFileSize).
// szFile is passed by reference.
//
// nszFileSize: The size that szFile was explicitly declared as.
//
// szFilter: Filter spec for dialog. In the form "\0\0\0". For example:
//
// "Text files (*.txt)\0*.txt\0\0"
//
// The description ("Text files (*.txt)" above) must be separated from the
// extension ("*.txt" above) by a "\0" character. The entire string must
// end in a double null ("\0\0").
//
// szDialogTitle: A string containig the title to display on the file browse dialog.
//
// szInitialDir: A string specifying the directory the browse dialog opens to.
//
// bMultiSel: Set to TRUE if you wish to enable multiple selection.
//
// listFiles: List that will be loaded with directory and filenames if multiple
// selection is enabled. List is passed by reference (by default,
// since list variables are pointers).
//
// bDerefLinks: Set to TRUE to dereference link files to the file browse.
//
// Return:
//
// Returns 0 when a file is successfully selected. Returns less than zero when
// when the user cancels/closes the browse dialog or an error occurs. If an
// error occurs, a message box displays the error identifier.
//
///////////////////////////////////////////////////////////////////////////////

function FileBrowseDlg( szFile, nszFileSize, szFilter, szDialogTitle, szInitialDir, bMultiSel, listFiles, bDerefLinks )
OPENFILENAME ofn;
STRING szMsg, szFileTitle[260];
STRING szCustomFilter[260], szTemp[260];
LONG nResult, n, nFlags, nErr;
STR260 str;
begin

UseDLL(WINSYSDIR ^ "comdlg32.dll");

nFlags = OFN_FILEMUSTEXIST | OFN_PATHMUSTEXIST | OFN_HIDEREADONLY |
OFN_NOCHANGEDIR | OFN_EXPLORER;
if bMultiSel then
nFlags = nFlags | OFN_ALLOWMULTISELECT;
endif;
if bDerefLinks = FALSE then
nFlags = nFlags | OFN_NODEREFERENCELINKS;
endif;

nResult = GetWindowHandle(HWND_INSTALL);

ofn.lStructSize = SizeOf(ofn);
ofn.hwndOwner = nResult;
// Notice how the address of an explicitly sized string
// is used when assigning to a member who was declared
// as a LONG string pointer (lpstr). For example, &szFilter.
ofn.lpstrFilter = &szFilter;
ofn.nFilterIndex = 1;

//The string pointed to by lpstrFile is modified by
//GetOpenFileName. The only way to have a string in
//script to reflect the change is to point lpstrFile
//to a structure that just a string member.
str.sz = szFile;
ofn.lpstrFile = &str;
ofn.nMaxFile = SizeOf(str);

ofn.lpstrFileTitle = &szFileTitle;
ofn.nMaxFileTitle = 260;
ofn.lpstrTitle = &szDialogTitle;
ofn.Flags = nFlags;
ofn.lpstrDefExt = &szTemp;
ofn.lpstrInitialDir = &szInitialDir;
ofn.hInstance = 0;
ofn.lpstrCustomFilter = &szCustomFilter;
ofn.nMaxCustFilter = 260;
ofn.lpfnHook = 0;

nResult = GetOpenFileNameA(&ofn);
if nResult = 1 then
if bMultiSel then
//A direct assignment in the form of szFile = str.sz
//will result in all data beyond the first null to be
//lost. This only happend when the string is assigned
//with a structure member. This is the reason why a
//very indirect method is being used the transfer the
//contents of str.sz to szFile
Resize(szFile, SizeOf(str));
RtlMoveMemory(szFile, &str, SizeOf(str));
StrGetTokens(listFiles, szFile, "");
else
szFile = str.sz;
endif;
else
nErr = CommDlgExtendedError();
switch (nErr)
case CDERR_DIALOGFAILURE: szMsg = CDERR_DIALOGFAILURE_MSG;
case CDERR_FINDRESFAILURE: szMsg = CDERR_FINDRESFAILURE_MSG;
case CDERR_INITIALIZATION: szMsg = CDERR_INITIALIZATION_MSG;
case CDERR_LOADRESFAILURE: szMsg = CDERR_LOADRESFAILURE_MSG;
case CDERR_LOADSTRFAILURE: szMsg = CDERR_LOADSTRFAILURE_MSG;
case CDERR_LOCKRESFAILURE: szMsg = CDERR_LOCKRESFAILURE_MSG;
case CDERR_MEMALLOCFAILURE: szMsg = CDERR_MEMALLOCFAILURE_MSG;
case CDERR_MEMLOCKFAILURE: szMsg = CDERR_MEMLOCKFAILURE_MSG;
case CDERR_NOHINSTANCE: szMsg = CDERR_NOHINSTANCE_MSG;
case CDERR_NOHOOK: szMsg = CDERR_NOHOOK_MSG;
case CDERR_NOTEMPLATE: szMsg = CDERR_NOTEMPLATE_MSG;
case CDERR_REGISTERMSGFAIL: szMsg = CDERR_REGISTERMSGFAIL_MSG;
case CDERR_STRUCTSIZE: szMsg = CDERR_STRUCTSIZE_MSG;
case FNERR_BUFFERTOOSMALL: szMsg = FNERR_BUFFERTOOSMALL_MSG;
case FNERR_INVALIDFILENAME: szMsg = FNERR_INVALIDFILENAME_MSG;
case FNERR_SUBCLASSFAILURE: szMsg = FNERR_SUBCLASSFAILURE_MSG;
endswitch;
if nErr != 0 then
// User did not close or cancel dialog box.
MessageBox("FileBrowseDlg() error:\n\n" + szMsg, SEVERE);
endif;
return -1;
endif;

UnUseDLL(WINSYSDIR ^ "comdlg32.dll");
return 0;
end;
#endif
Use below code in your setup.rul file.

Click on your setup.rul file to bring up your script in the right hand window and insert the following code to your script, after the #include "ifx.h" statement.

#include "brwsdlg.h"
#include "brwsdlg.rul"

Add the following sample script to call the API to display the file browse dialog.
function OnBegin()
// FileBrowseDlg requires the szFile parameter be explicitly sized
// and that the size be passed as the second parameter.
STRING szFile[512], svDir, svFileList, svTemp;
NUMBER nResult, nReturn;
BOOL bMultiSel, bDerefLinks;
LIST listFiles;

begin
// If I want to support multiple selection, set bMultiSel to TRUE
// and pass in a valid string list.
bMultiSel = TRUE;
bDerefLinks = FALSE;
listFiles = ListCreate(STRINGLIST);
// Open the file browse dialog.
nResult = FileBrowseDlg( szFile, 512, "All files (*.*)\0*.txt\0\0", "Select File", "c:\\", bMultiSel, listFiles, bDerefLinks );
if nResult = 0 then
if bMultiSel then
// If I chose multiple selection, I must parse the info, which is stored
// in list. First item will be dir, all others are individual filenames.
nReturn = ListGetFirstString(listFiles, svTemp);
while nReturn != END_OF_LIST
svFileList = svFileList + svTemp + "\n";
nReturn = ListGetNextString(listFiles, svTemp);
endwhile;
MessageBox("Directory (first item) and selected files:\n\n" + svFileList, 0);
else
// No multiple selection, so a single file/path was set.
MessageBox("Selected file:\n\n" + szFile, 0);
endif;
endif;
ListDestroy(listFiles);
end;

Note: You have to customize this code according to your needs.

Installshield custom dialog, custom actions, and code

Wednesday, April 15, 2009

Creating custom file browse dialog

1 comment: