The Script - Kitabsiz.az

Transcription

ChrisF.A.JohnsonandJayantVarma
ShellScriptingRecipes
AProblem-SolutionApproach
SecondEdition
ChrisF.A.Johnson
JayantVarma
Anysourcecodeorothersupplementarymaterialreferencedbythe
authorinthistextisavailabletoreadersat
www.apress.com/9781484202210.Fordetailedinformationabout
howtolocateyourbook’ssourcecode,goto
www.apress.com/source-code/.Readerscanalsoaccesssourcecode
atSpringerLinkintheSupplementaryMaterialsectionforeach
chapter.
ISBN978-1-4842-0221-0
e-ISBN978-1-4842-0220-3
DOI10.1007/978-1-4842-0220-3
©Apress2015
ShellScriptingRecipes,SecondEdition
ManagingDirector:WelmoedSpahr
LeadEditor:LouiseCorrigan
EditorialBoard:SteveAnglin,LouiseCorrigan,JimDeWolf,
JonathanGennick,RobertHutchinson,MichelleLowman,James
Markham,SusanMcDermott,MatthewMoodie,JeffreyPepper,
DouglasPundick,BenRenow-Clarke,DominicShakeshaft,
GwenanSpearing,MattWade,SteveWeiss
CoordinatingEditor:MarkPowers
Compositor:SPiGlobal
Indexer:SPiGlobal
Artist:SPiGlobal
Forinformationontranslations,pleasee-mail
[email protected],orvisitwww.apress.com.
ApressandfriendsofEDbooksmaybepurchasedinbulkfor
academic,corporate,orpromotionaluse.eBookversionsand
licensesarealsoavailableformosttitles.Formoreinformation,
referenceourSpecialBulkSales–eBookLicensingwebpageat
www.apress.com/bulk-sales.
Thisworkissubjecttocopyright.Allrightsarereservedbythe
Publisher,whetherthewholeorpartofthematerialisconcerned,
specificallytherightsoftranslation,reprinting,reuseof
illustrations,recitation,broadcasting,reproductiononmicrofilmsor
inanyotherphysicalway,andtransmissionorinformationstorage
andretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Exemptedfromthislegalreservationarebriefexcerptsin
connectionwithreviewsorscholarlyanalysisormaterialsupplied
specificallyforthepurposeofbeingenteredandexecutedona
computersystem,forexclusiveusebythepurchaserofthework.
Duplicationofthispublicationorpartsthereofispermittedonly
undertheprovisionsoftheCopyrightLawofthePublisher’s
location,initscurrentversion,andpermissionforusemustalways
beobtainedfromSpringer.Permissionsforusemaybeobtained
throughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Trademarkednames,logos,andimagesmayappearinthisbook.
Ratherthanuseatrademarksymbolwitheveryoccurrenceofa
trademarkedname,logo,orimageweusethenames,logos,and
imagesonlyinaneditorialfashionandtothebenefitofthe
trademarkowner,withnointentionofinfringementofthe
trademark.Theuseinthispublicationoftradenames,trademarks,
servicemarks,andsimilarterms,eveniftheyarenotidentifiedas
such,isnottobetakenasanexpressionofopinionastowhetheror
nottheyaresubjecttoproprietaryrights.
Whiletheadviceandinformationinthisbookarebelievedtobe
trueandaccurateatthedateofpublication,neithertheauthorsnor
theeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesno
warranty,expressorimplied,withrespecttothematerialcontained
herein.
DistributedtothebooktradeworldwidebySpringer
Science+BusinessMediaNewYork,233SpringStreet,6thFloor,
NewYork,NY10013.Phone1-800-SPRINGER,fax(201)3484505,[email protected],orvisit
www.springeronline.com.ApressMedia,LLCisaCaliforniaLLC
andthesolemember(owner)isSpringerScience+BusinessMedia
FinanceInc(SSBMFinanceInc).SSBMFinanceIncisaDelaware
corporation.
Thisbookisdedicatedtomyparents,whowouldhavebeenquite
proudtoseethisbook.
—JayantVarma
Acknowledgments
IwouldliketothankthewonderfulstaffatApressforthe
opportunitytoupdatethisbook.SpecialthankstoLouiseandMark,
whofacilitatedthequickturnaroundonthebookandgettingitto
print.Lastly,specialthankstomyfamilyfortheirsupportingetting
thisbookcompleted.
—JayantVarma
Contents
Chapter1:ThePOSIXShellandCommand-LineUtilities
ShellCommands
echo
printf
set
shift
type
getopts
case
eval
local
ParametersandVariables
PositionalParameters
SpecialParameters
standard-vars—ACollectionofUsefulVariables
Patterns
PathnameExpansion
RegularExpressions
ParameterExpansion
TheBourneShellExpansions
POSIXParameterExpansions
Shell-SpecificExpansions,bash2,andksh93
ShellArithmetic
Aliases
SourcingaFile
Functions
FunctionsAreFast
CommandSubstitutionIsSlow
UsingtheFunctionsinThisBook
standard-funcs:ACollectionofUsefulCommands
1.1get_key—GetaSingleKeystrokefromtheUser
1.2getline—PromptUsertoEnteraLine
1.3press_any_key—PromptforaSingleKeypress
1.4menu1—PrintaMenuandExecuteaSelectedCommand
1.5arg—PromptforRequiredArgumentIfNoneSupplied
1.6die—PrintErrorMessageandExitwithErrorStatus
1.7show_date—DisplayDateinD[D]MMMYYYYFormat
1.8date_vars—SetDateandTimeVariables
1.9is_num—IsThisaPositiveInteger?
1.10abbrev_num—AbbreviateLargeNumbers
1.11commas—AddThousandsSeparatorstoaNumber
1.12pr1—PrintArguments,OnetoaLine
1.13checkdirs—CheckforDirectories;CreateIfNecessary
1.14checkfiles—CheckThataDirectoryContainsCertainFiles
1.15zpad—PadaNumberwithLeadingZeroes
1.16cleanup—RemoveTemporaryFilesandResetTerminalon
Exit
TheUnixUtilities
cat:ConcatenateFilestotheStandardOutput
sed:ATextStreamEditor
awk:PatternScanningandProcessingLanguage
grep:PrintLinesMatchingaRegularExpression
date:ShoworSettheSystemDate
tr:ACharacterTranslationUtility
wc:CountCharacters,Words,andLinesinaFile
file:DeterminetheFileType
ls:SortandProvideDetailsAboutFiles
uniq:RemoveConsecutiveDuplicateLines
sudo:ExecuteCommandsastheSuperuser
split:DivideaFileintoEqual-SizedPieces
which:ShowtheFullPathtoaCommand
gs,gv:Render,Convert,orViewPostScriptandPDFFiles
Summary
Chapter2:PlayingwithFiles:Viewing,Manipulating,and
EditingTextFiles...
EndoftheLineforOSDifferences:ConvertingTextFiles
2.1dos2unix—ConvertWindowsTextFilestoUnix
2.2unix2dos—ConvertaUnixFiletoWindows
2.3mac2unix—ConvertMacintoshFilestoUnix
2.4unix2mac—ConvertUnixFilestoMacFormat
2.5dos2mac—ConvertWindowsFilestoMacintosh
2.6mac2dos—ConvertMacintoshFilestoWindows
DisplayingFiles
2.7prn—PrintFilewithLineNumbers
2.8prw—PrintOneWordperLine
2.9wbl—SortWordsbyLength
FormattingFileFacts
2.10finfo—FileInformationDisplay
2.11wfreq—WordFrequency
2.12lfreq—LetterFrequency
2.13fed—ASimpleBatchFileEditor
Summary
Chapter3:StringBriefs
CharacterActions:Thechar-funcsLibrary
3.1chr—ConvertaDecimalNumbertoanASCIICharacter
3.2asc—ConvertaCharactertoItsDecimalEquivalent
3.3nxt—GettheNextCharacterinASCIISequence
3.4upr—ConvertCharacter(s)toUppercase
3.5lwr—ConvertCharacter(s)toLowercase
StringCleaning:Thestring-funcsLibrary
3.6sub—ReplaceFirstOccurrenceofaPattern
3.7gsub—GloballyReplaceaPatterninaString
3.8repeat—BuildaStringofaSpecifiedLength
3.9index,rindex—FindPositionofOneStringWithinAnother
3.10substr—ExtractaPortionofaString
3.11insert_str—PlaceOneStringInsideAnother
Summary
Chapter4:What’sinaWord?
FindingandMassagingWordLists
wf-funcs:WordFinderFunctionLibrary
4.1write_config—WriteUser’sInformationtothe
ConfigurationFile
4.2do_config—CheckForandSourceDefaultConfiguration
File
4.3set_sysdict—SelecttheDictionaryDirectory
4.4mkwsig—SortLettersinaWord
4.5wf-clean—RemoveCarriageReturnsandAccents
4.6wf-compounds—SquishCompoundWordsandSavewith
Lengths
4.7wf-setup—PrepareWordandAnagramLists
PlayingwithMatches
4.8wf—FindWordsThatMatchaPattern
4.9wfb—FindWordsThatBeginwithaGivenPattern
4.10wfe—FindWordsThatEndwithaGivenPattern
4.11wfc—FindWordsThatContainaGivenPattern
4.12wfit—FindWordsThatFitTogetherinaGrid
4.13anagram—FindWordsThatAreAnagramsofaGiven
Word
4.14aplus—FindAnagramsofaWordwithaLetterAdded
4.15aminus—RemoveEachLetterinTurnandAnagram
What’sLeft
Summary
Chapter5:ScriptingbyNumbers
Themath-funcsLibrary
5.1fpmul—MultiplyDecimalFractions
5.2int—ReturntheIntegerPortionofaDecimalFraction
5.3round—RoundtheArgumenttotheNearestInteger
5.4pow—RaiseaNumbertoAnyGivenPower
5.5square—RaiseaNumbertotheSecondPower
5.6cube—RaiseaNumbertotheThirdPower
5.7calc—ASimpleCommand-LineCalculator
AddingandAveraging
5.8total—AddaListofNumbers
5.9mean—FindtheArithmeticMeanofaListofNumbers
5.10median—FindtheMedianofaListofNumbers
5.11mode—FindtheNumberThatAppearsMostinaList
5.12range—FindtheRangeofaSetofNumbers
5.13stdev—FindingtheStandardDeviation
ConvertingBetweenUnitSystems
5.14conversion-funcs—ConvertingMetricUnits
5.15conversion—AMenuSystemforMetricConversion
Summary
Chapter6:LooseNamesSinkScripts:BringingSanityto
Filenames
What’sinaName?
POSIXPortableFilenames
OKFilenames
FunctioningFilenames:Thefilename-funcsLibrary
6.1basename—ExtracttheLastElementofaPathname
6.2dirname—ReturnAllbuttheLastElementofaPathname
6.3is_pfname—CheckforPOSIXPortableFilename
6.4is_OKfname—CheckWhetheraFilenameIsAcceptable
6.5pfname—ConvertNonportableCharactersinFilename
6.6OKfname—MakeaFilenameAcceptable
6.7is_whitespc—DoestheFilenameContainWhitespace
Characters?
6.8whitespc—FixFilenamesContainingWhitespaceCharacters
6.9is_dir—IsThisaDirectoryISeeBeforeMe?
6.10nodoublechar—RemoveDuplicateCharactersfroma
String
6.11new_filename—ChangeFilenametoDesiredCharacterSet
6.12fix_pwd—FixAlltheFilenamesintheCurrentDirectory
6.13fixfname—ConvertFilenamestoSensibleNames
Summary
Chapter7:TreadingaRighteousPATH
Thepath-funcsLibrary
7.1path—DisplayaUser’sPATH
7.2unslash—RemoveExtraneousSlashes
7.3checkpath—CleanUpthePATHVariable
7.4addpath—AddDirectoriestothePATHVariable
7.5rmpath—RemoveOneorMoreDirectoriesfrom$PATH
Summary
Chapter8:TheDatingGame
Thedate-funcsLibrary
8.1split_date—DivideaDateintoDay,Month,andYear
8.2is_leap_year—IsThereanExtraDayThisYear?
8.3days_in_month—HowManyDaysHathSeptember?
8.4JulianDates
8.5dateshift—AddorSubtractaNumberofDays
8.6diffdate—FindtheNumberofDaysBetweenTwoDates
8.7day_of_week—FindtheDayoftheWeekforAnyDate
8.8display_date—ShowaDateinTextFormat
8.9parse_date—DecipherVariousFormsofDateString
8.10valid_date—WhereWasIonNovember31st?
Summary
Chapter9:GoodHousekeeping:MonitoringandTidyingUp
FileSystems
9.1dfcmp—NotifyUserofMajorChangesinDiskUsage
HowItWorks
9.2symfix—RemoveBrokenSymbolicLinks
HowItWorks
9.3sym2file—ConvertsSymbolicLinkstoRegularFiles
HowItWorks
9.4zrm—RemoveEmptyFiles
HowItWorks
9.5undup—RemoveDuplicateFiles
HowItWorks
9.6lsr—ListtheMostRecent(orOldest)FilesinaDirectory
HowItWorks
Summary
Chapter10:Screenplay:Thescreen–funcsLibrary
10.1screen-vars—VariablesforScreenManipulation
HowItWorks
10.2set_attr—SetScreen-PrintingAttributes
HowItWorks
10.3set_fg,set_bg,set_fgbg—SetColorsforPrintingtothe
Screen
HowItWorks
10.4cls—CleartheScreen
HowItWorks
Notes
10.5printat—PositionCursorbyRowandColumn
HowItWorks
Notes
10.6put_block_at—PrintLinesinaColumnAnywhereonthe
Screen
HowItWorks
10.7get_size—SetLINESandCOLUMNSVariables
HowItWorks
10.8max_length—FindtheLengthoftheLongestArgument
HowItWorks
10.9print_block_at—PrintaBlockofLinesAnywhereonthe
Screen
HowItWorks
10.10vbar,hbar—PrintaVerticalorHorizontalBar
HowItWorks
10.11center—CenteraStringonNColumns
HowItWorks
10.12flush_right—AlignStringwiththeRightMargin
HowItWorks
10.13ruler—DrawaRulerAcrosstheWidthandHeightofthe
Window
HowItWorks
10.14box_block,box_block_at—PrintTextSurroundedbya
Box
HowItWorks
10.15clear_area,clear_area_at—ClearanAreaoftheScreen
HowItWorks
10.16box_area,box_area_at—DrawaBoxAroundanArea
HowItWorks
10.17screen-demo—SavingandRedisplayingAreasofthe
Screen
HowItWorks
Summary
Chapter11:Aging,Archiving,andDeletingFiles
11.1date-file—AddaDatestamptoaFilename
HowItWorks
Notes
11.2rmold—RemoveOldFiles
HowItWorks
11.3keepnewest—RemoveAllbuttheNewestorOldestFiles
HowItWorks
Summary
Chapter12:CoveringAllYourDatabases
12.1lookup—FindtheCorrespondingValueforaKey
HowItWorks
shdb-funcs:ShellDatabaseFunctionLibrary
12.2load_db—ImportDatabaseintoShellArray
HowItWorks
Notes
12.3split_record—SplitaRecordintoFields
HowItWorks
12.4csv_split—ExtractFieldsfromCSVRecords
HowItWorks
12.5put_record—AssembleaDatabaseRecordfromanArray
HowItWorks
12.6put_csv—AssembleFieldsintoaCSVRecord
HowItWorks
12.7db-demo—ViewandEditaPasswordFile
HowItWorks
PhoneBase:ASimplePhoneNumberDatabase
HowItWorks
12.8ph—LookUpaPhoneNumber
12.9phadd—AddanEntrytoPhoneBase
12.10phdel—DeleteanEntryfromPhoneBase
12.11phx—ShowphSearchResultsinanXWindow
Summary
Chapter13:HomeontheWeb
PlayingwithHypertext:Thehtml-funcsLibrary
13.1get_element—ExtracttheFirstOccurrenceofanElement
13.2split_tags—PutEachTagonItsOwnLine
13.3html-title—GettheTitlefromanHTMLFile
HTMLontheFly:Thecgi-funcsLibrary
13.4x2d2—ConvertaTwo-DigitHexadecimalNumberto
Decimal
13.5dehex—ConvertHexStrings(%XX)toCharacters
13.6filedate—FindandFormattheModificationDateofaFile
CreatingHTMLFiles
13.7mk-htmlindex—CreateanHTMLIndex
13.8pretext—CreateaWrapperAroundaTextFile
13.9text2html—ConvertaTextFiletoHTML
13.10demo.cgi—ACGIScript
Summary
Chapter14:TakingCareofBusiness
14.1prcalc—APrintingCalculator
HowItWorks
14.2gle—KeepingRecordsWithoutaShoebox
HowItWorks
Summary
Chapter15:RandomActsofScripting
Therand-funcsLibrary
15.1random—ReturnOneorMoreRandomIntegersina
GivenRange
HowItWorks
Notes
15.2toss—SimulateTossingaCoin
HowItWorks
15.3randstr—SelectaStringatRandom
HowItWorks
ARandomSamplingofScripts
15.4rand-date—GenerateRandomDatesinISOFormat
HowItWorks
Notes
15.5randsort—PrintLinesinRandomOrder
HowItWorks
15.6randomword—GenerateRandomWordsAccordingto
FormatSpecifications
HowItWorks
15.7dice—RollaSetofDice
HowItWorks
15.8throw—ThrowaPairofDice
HowItWorks
Summary
Chapter16:ASmorgasbordofScripts
16.1topntail—RemoveTopandBottomLinesfromaFile
HowItWorks
16.2flocate—LocateFilesbyFilenameAlone
HowItWorks
16.3sus—DisplayaPOSIXManPage
HowItWorks
16.4cwbw—CountWordsBeginningWith
HowItWorks
Notes
16.5cci—Configure,Compile,andInstallfromTarball
HowItWorks
Notes
16.6ipaddr—FindaComputer’sNetworkAddress
HowItWorks
16.7ipaddr.cgi—PrinttheRemoteAddressofanHTTP
Connection
HowItWorks
16.8iprev—ReversetheOrderofDigitsinanIPAddress
HowItWorks
Notes
16.9intersperse—InsertaStringBetweenDoubledCharacters
HowItWorks
16.10ll—UseaPagerforaDirectoryListingOnlyIfNecessary
HowItWorks
16.11name-split—DivideaPerson’sFullNameintoFirst,Last,
andMiddleNames
HowItWorks
16.12rot13—EncodeorDecodeText
HowItWorks
Notes
16.13showfstab—ShowInformationfrom/etc/fstab
HowItWorks
Notes
16.14unique—RemoveAllDuplicateLinesfromaFile
HowItWorks
Summary
Chapter17:ScriptDevelopmentManagement
17.1script-setup—PreparetheScriptingEnvironment
HowItWorks
Notes
17.2cpsh—InstallScriptandMakeBackupCopy
HowItWorks
17.3shgrep—SearchScriptsforStringorRegularExpression
HowItWorks
17.4shcat—DisplayaShellScript
HowItWorks
Summary
AppendixA:InternetScriptingResources
IntroductionstoShellScripting
IntermediateandAdvancedScripting
CollectionsofScripts
HomePagesforShells
RegularExpressions,sed,andawk
MiscellaneousPages
HistoryoftheShell
Index
ContentsataGlance
AbouttheAuthors
Acknowledgments
Chapter1:ThePOSIXShellandCommand-LineUtilities
Chapter2:PlayingwithFiles:Viewing,Manipulating,andEditing
TextFiles
Chapter6:LooseNamesSinkScripts:BringingSanitytoFilenames
Chapter7:TreadingaRighteousPATH
Chapter9:GoodHousekeeping:MonitoringandTidyingUpFile
Systems
Chapter10:Screenplay:Thescreen–funcsLibrary
Chapter11:Aging,Archiving,andDeletingFiles
Chapter12:CoveringAllYourDatabases
Chapter15:RandomActsofScripting
Chapter16:ASmorgasbordofScripts
Chapter17:ScriptDevelopmentManagement
AppendixA:InternetScriptingResources
Index
AbouttheAuthors
ChrisF.A.Johnson
wasintroducedtoUnixin1990andlearnedshellscriptingbecause
therewasnoCcompileronthesystem.Hisfirstmajorprojectwasa
menu-driven,user-extensibledatabasesystemwithreportgenerator.
Chrisusestheshellashisprimary,general-purposeprogramming
language,andhisprojectshaveincludedamemberdatabase,
menuingsystem,andPOP3mailfilteringandretrieval.Chrisisthe
coauthorofProBashProgramming(Apress,2015).Whennot
pushingshellscriptingtothelimit,hedesignsandcodeswebsites,
teacheschess,andcomposescrypticcrosswords.
JayantVarma
isthefounderofOZApps(www.oz-apps.com),aconsulting,
training,anddevelopmentcompanyprovidingITsolutions
(specializationinmobiletechnology).Heisanexperienced
developerwithmorethan20yearsofindustryexperiencespread
acrossseveralcountries.Heistheauthorofanumberofbookson
iOSdevelopment,includingLearnLuaforiOSGameDevelopment
(Apress,2012),Xcode6Essentials(Packt,2015),MoreiPhone
DevelopmentwithSwift(Apress,2015),andMoreiPhone
DevelopmentwithObjective-C(Apress,2015).Hehasalsobeena
universitylecturerinAustraliawherehecurrentlyresides.Heloves
travelingandfindsEuropetobehisfavoritedestination.
©ChrisF.A.JohnsonandJayantVarma2015
ChrisF.A.JohnsonandJayantVarma,ShellScriptingRecipes,DOI10.1007/978-1-48420220-3_1
Chapter1:ThePOSIXShelland
Command-LineUtilities
ChrisF.A.Johnson1 andJayantVarma1
(1) Ontario,Canada
ThePOSIXshellisadescendantoftheKornShell,whichisa
descendantoftheBourneshell.Thebasicsyntaxhasremainedthe
same,andBourneshellscriptswillusuallyrunsuccessfullyina
POSIXshell.NotalloftheKornShellfeaturesareincludedin
POSIX(therearenoarrays,forexample),butthemostimportant
onesare.Those,whichwouldbeimportanttomanydevelopers,are
stringmanipulationandarithmetic.
Thescriptsinthisbookmakeextensiveuseoffeaturesofthe
POSIXshell,andkeepexternalcommandstoaminimum.This
chapterpresentsanoverviewofthefeaturesofthePOSIXshelland
theexternalUnixcommandsusedinthisbook,withoutgoinginto
greatdetail.Furtherinformationisavailableinthedocumentation
fortheshellandthevariouscommandsaswellasonmanyweb
pages,anumberofwhicharelistedintheAppendix.Wewillalso
explainsomeoftheidiosyncrasiesusedinthescripts,andpresenta
libraryoffunctionsthatareusedbymanyscripts.
ShellCommands
Thesedescriptionsarebriefoverviewsofthebuilt-incommands;
foracompletedescription,seeyourshell’smanpage.
echo
Theechocommandprintsitsargumentsseparatedbysinglespaces
followedbyanewline.Ifanunquotedvariablecontainscharacters
presentin$IFS(see“ParametersandVariables”laterinthis
chapter),thenthevariablewillbesplitonthosecharacters:
$list="abcdefgh"
$echo$list
abcdefgh
Ifthevariableisquoted,allinternalcharacterswillbe
preserved:
$echo"$list"
abcdefgh
IntheearlydaysofUnix,twodifferentversionsofecho
appeared.Oneversionconvertedescapesequences,suchas\tand
\n,intothecharacterstheyrepresentintheClanguage;\c
suppressedthenewline,anddiscardedanyfurthercharacters.The
otherusedthe-noptiontosuppressthetrailingnewlineanddidnot
convertescapesequences.ThePOSIXstandardforechosaysthat
“Implementationsshallnotsupportanyoptions”and“Ifthefirst
operandis-n,orifanyoftheoperandscontainabackslash(‘\’)
character,theresultsareimplementation-defined.”Inotherwords,
youcannotrelyonecho’sbehaviorbeingoneortheother.Itisbest
nottouseechounlessyouknowexactlywhatechoisgoingtoprint,
andyouknowthatitwillnotcontainanyproblemcharacters.The
preferredcommandisprintf.
printf
Thiscommandmaybebuiltintotheshellitself,oritmaybean
externalcommand.LiketheC-languagefunctiononwhichitis
based,printftakesaformatoperandthatdescribeshowthe
remainingargumentsaretobeprinted,andanynumberofoptional
arguments.Theformatstringmaycontainliteralcharacters,escape
sequences,andconversionspecifiers.Escapesequences(themost
commononesbeing\nfornewline,\tfortab,and\rforcarriage
return)informatwillbeconvertedtotheirrespectivecharacters.
Conversionspecifiers,%s,%b,%d,%x,and%o,arereplacedbythe
correspondingargumentonthecommandline.Some
implementationssupportotherspecifiers,buttheyarenotusedin
thisbook.Whentherearemoreargumentsthanspec-ifiers,the
formatstringisreuseduntilalltheargumentshavebeenconsumed.
The%sspecifierinterpretsitsargumentasastringandprintsit
literally:
$printf"%s\n""qwer\ty"1234+5678
qwer\ty
1234+5678
The%bspecifierislike%s,butconvertsescapesequencesinthe
argument:
$printf"%b\n""qwer\ty""asdf\nghj"
qwery
asdf
ghj
The%d,%x,and%ospecifiersprinttheirargumentsasdecimal,
hexadecimal,andoctalnumbers,respectively.
$printf"%d%x%o\n"151515
15f17
Theconversionspecifiersmaybeprecededbyflagsforwidth
specification,optionallyprecededbyaminussignindicatingthat
theconversionistobeprintedflushleft,insteadofflushright,in
thespecifiednumberofcolumns:
$printf"%7d:\n%7s:\n%-7s:\n"23Cord
Auburn
23:
Cord:
Auburn:
Inanumericfield,a0beforethewidthflagindicatespadding
withzeroes:
$printf"%07d\n"13
0000013
set
IntheOxfordEnglishDictionary,thelongestentryisfortheword
set—thirty-twopagesinmyCompactEdition.IntheUnixshell,the
setcommandisreallythreecommandsinone.Withoutany
arguments,itprintsthenamesandvaluesofallshellvariables
(includingfunctions).Withoneormoreoptionarguments,italters
theshell’sbehavior.Anynon-optionargumentsareplacedinthe
positionalparameters.
Onlythreeoptionstosetareusedinthisbook:
-v:Printshellinputlinesastheyareread.
-x:Printcommandsandtheirargumentsastheyareexecuted.
-f:Disablefilenamegeneration(globbing).
Giventhisscript,whichiscallxx.sh,
echo"Numberofpositionalparameters:$#"
echo"Firstparameter:${1:-EMPTY}"
shift$(($#-1))
echo"Lastparameter:${1:-EMPTY}"
itsoutputisthis:
$xx.shthequickbrownfox
Numberofpositionalparameters:4
Firstparameter:the
Lastparameter:fox
Ifset-visaddedtothetopofthescript,andthestandard
outputredirectedtooblivion,thescriptitselfisprinted:
$xx.shthequickbrownfox>/dev/null
echo"Numberofpositionalparameters:$#"
echo"Firstparameter:${1:-EMPTY}"
shift$(($#-1))
echo"Lastparameter:${1:-EMPTY}"
Ifset-visreplacedwithset-x,variablesandarithmetic
expressionsarereplacedbytheirvalueswhenthelinesareprinted;
thisisausefuldebuggingtool:
$xx.shthequickbrownfox>/dev/null
++echo'Numberofpositionalparameters:
4'
++echo'Firstparameter:the'
++shift3
++echo'Lastparameter:fox'
++exit
Todemonstratetheset-foption,andtheuseof+toreversethe
operation,runthefollowingscriptinanemptydirectory:
##Createanumberoffilesusingbrace
expansion(bash,ksh)
touch
{a,b,c,d}${RANDOM}_{e,r,g,h}${RANDOM}
##Turnofffilenameexpansion
set-f
printf"%-22s%-22s%-22s\n"*;echo##
Displayasterisk
printf"%-22s%-22s%-22s\n"*h*;echo##
Display"*h*"
##Turnfilenameexpansionbackon
set+f
printf"%-22s%-22s%-22s\n"*;echo##
Printallfilenames
printf"%-22s%-22s%-22s\n"*h*;echo##
Printfilenamescontaining"h"
Whenthescriptisrun,thisistheoutput:
$xx.sh
*
*h*
a12603_e6243a28923_h23375a29140_r28413
a5760_g7221b17774_r4121b18259_g11343
b18881_e10656b660_h32228c22841_r19358
c26906_h14133c29993_g6498c6576_e25837
d11453_h12972d25162_e3276d7984_r25591
d8972_g31551
a28923_h23375b660_h32228c26906_h14133
d11453_h12972
Youcanusesettosplitstringsintopiecesbychangingthe
valueof$IFS.Forexample,tosplitadate,whichcouldbe2005-0301or2003/09/29or2001.01.01,$IFScanbesettoallthepossible
charactersthatcouldbeusedasseparators.Theshellwillperform
wordsplittingonanycharactercontainedin$IFS:
$IFS='-/.'
$set2005-03-01
$printf"%s\n""$@"
2005
03
01
Whenthevaluetobesetiscontainedinavariable,adouble
dashshouldbeusedtoensurethatthevalueofthevariableisnot
takentobeanoption:
$var="-f-x-o"
$set--$var
shift
Theleadingpositionalparametersareremoved,andtheremaining
parametersaremovedup.Bydefault,oneparameterisremoved,but
anargumentmayspecifymore:
$set12345678
$echo"$*($#)"
12345678(8)
$shift
$echo"$*($#)"
2345678(7)
$shift3
$echo"$*($#)"
5678(4)
Tip
Thisismostusefulwhenusedinscriptstoiteratethrougha
numberofparameterspassed.Accessthe$0parameterandkeep
shiftingtilltherearenomoretogothrough.Howevershift
removestheparameters,sobeawareifyouwanttoreusethe
parameters.
Someshellswillcomplainiftheargumenttoshiftislargerthanthe
numberofpositionalparameters.
type
ThePOSIXstandardsaystype“shallindicatehoweachargument
wouldbeinterpretedifusedasacommandname.”Itsreturnstatus
maybeusedtodeterminewhetheracommandisavailable:
iftypestat>/dev/null2>&1##discard
theoutput
then
stat"$file"
fi
Ifthecommandisanexecutablefile,typeprintsthepathtothe
file;otherwise,itprintsthetypeofcommandthatwillbeinvoked:
function,alias,orshellbuiltin.Itsoutputisnotstandardacross
differentshells,andthereforecannotbeusedreliablyinashell
script.
Thefourargumentstotypeinthefollowingexamplerepresent
anexecutablefile,afunction,anonexistentcommand,andanalias.
$typelspr1grebecoh
lsishashed(/bin/ls)
pr1isafunction
pr1()
{
case$1in
-w)
pr_w=
;;
*)
pr_w=-.${COLUMNS:-80}
;;
esac;
printf"%${pr_w}s\n""$@"
}
bash:type:greb:notfound
ecohisaliasedtoècho'
Unlikemostshells,bashwillprintthedefinitionofafunction.
getopts
Thecommandgetoptsparsesthepositionalparametersaccordingto
astringofacceptableoptions.Ifanoptionisfollowedbyacolon,
anargumentisexpectedforthatoption,andwillbestoredin
$OPTARG.Thisexampleaccepts-a,-b,and-c,with-bexpectingan
argument:
whilegetoptsab:copt
do
case$optin
a)echo"Option-afound";;
b)echo"Option-bfoundwithargument$OPTARG"
;;
c)echo"Option-cfound";;
*)echo"Invalidoption:$opt";exit5;;
esac
done
case
Aworkhorseamongtheshell’sbuilt-incommands,caseallows
multiplebranches,andistheidealtool,ratherthangrep,for
determiningwhetherastringcontainsapatternormultiplepatterns.
Theformatis
caseSTRINGin
PATTERN[|PATTERN...])[list];;
[PATTERN[|PATTERN...])[list];;...]
esac
ThePATTERNisapathnameexpansionpattern,notaregular
expression,andthelistofcommandsfollowingthefirstPATTERN
thatmatchesisexecuted.(Seethe“Patterns”sectionfurtheronfor
anexplanationofthetwotypesofpatternmatching.)
eval
Thecommandevalcausestheshelltoevaluatetherestoftheline,
thenexecutetheresult.Inotherwords,itmakestwopassesatthe
commandline.Forexample,giventhecommand:
eval"echo\${$#}"
Thefirstpasswillgenerateecho${4}(assumingthatthereare4
positionalparameters).Thiswillthenprintthevalueofthelast
positionalparameter,$4.
local
Thelocalcommandisusedinfunctions;ittakesoneormore
variablesasargumentsandmakesthoselocaltothefunctionandits
children.ThoughnotpartofthePOSIXstandard,itisbuiltinto
manyshells;bashandtheashfamilyhaveit,andpdkshhasitasa
standardaliasfortypeset(whichisalsonotincludedinPOSIX).In
KornShell93(generallyreferredtoasksh93),ifafunctionis
definedintheportablemanner(asusedthroughoutthisbook),there
isnowaytomakeavariablelocaltoafunction.
Inthisbook,localisusedonlyinthefewscriptsthatare
writtenspecificallyforbash,mostoftenforsetting$IFSwithout
havingtorestoreittoitsoriginalvalue:
localIFS=$NL
ParametersandVariables
Parametersarenamesusedtorepresentinformation;therearethree
classesofparameters:Positionalparametersarethecommand-line
arguments,andarenumberedbeginningwith$1;variablesare
parametersdenotedbyanamethatcontainsonlyletters,numbers
andunderscores,andthatbeginswithaletteroranunderscore;and
specialparametersthatarerepresentedbynon-alphanumeric
characters.
PositionalParameters
Positionalparametersarethecommand-lineargumentspassedtoa
scriptorafunction,andarenumberedbeginningwith1.Parameters
greaterthen9mustbeenclosedinbraces:${12}.Thisistopreserve
compatibilitywiththeBourneshell,whichcouldonlyaccessthe
firstninepositionalparameters;$12representsthecontentsof$1,
followedbythenumber2.Thepositionalparameterscanbe
assignednewvalues,withthesetcommand.(Seetheexample
under“SpecialParameters.”)
SpecialParameters
Theparameters$*and$@expandtoallthepositionalparameters,
and#representsthenumberofpositionalparameters.Thisfunction
demonstratesthefeaturesoftheseparameters:
demo()
{
printf"Numberofparameters:%d\n"$#
printf"Thefirstparameter:%s\n""$1"
printf"Thesecondparameter:%s\n""$2"
printf"\nAlltheparameters,eachonaseparate
line:\n"
printf"\t%s\n""$@"
printf"\nAlltheparameters,ononeline:\n"
printf"\t%s\n""$*"
printf"\nEachwordintheparametersonitsown
line:\n"
printf"\t%s\n"$*
}
Here,thedemofunctionisrunwiththreearguments:
$demoThe"quickbrown"fox
Numberofparameters:3
Thefirstparameter:The
Thesecondparameter:quickbrown
Alltheparameters,eachonaseparate
line:
The
quickbrown
fox
Alltheparameters,ononeline:
Thequickbrownfox
Eachwordintheparametersonitsown
line:
The
quick
brown
fox
Thedecimalexitcodeofthepreviouscommandexecuted(0for
success,non-zeroforfailure)isstoredin$?:
$true;echo$?
0
$false;echo$?
1
Theshell’scurrentoptionflagsarestoredin$-;theshell’s
processIDisin$$;$!istheprocessIDofthemostrecently
executedbackgroundcommand,and$0isthenameofthecurrent
shellorscript:
$sleep4&
[1]12725
$printf"PID:%d\nBackgroundcommandPID:
%d\n"$$$!
PID:12532
BackgroundcommandPID:12725
$printf"Currentlyexecuting%swith
options:%s\n""$0""$-"
Currentlyexecutingbashwithoptions:
fhimBH
ShellVariables
Thesearethevariablesthatareassignedvaluesatthecommandline
orinascript.Thesystemortheshellitselfalsosetanumberof
variables;thoseusedinthisbookare
$HOME:Thepathnameofuser’shomedirectory(e.g.,
/home/chris).
$IFS:Alistofcharactersusedasinternalfieldseparatorsfor
wordsplittingbytheshell.Thedefaultcharactersarespace,
tab,andnewline.Stringsofcharacterscanbebrokenupby
changingthevalueof$IFS:
$IFS=-;date=2005-04-11;
printf"%s\n"$date
2005
04
11
$PATH:Thiscolon-separatedlistofdirectoriestellstheshell
whichdirectoriestosearchforacommand.Toexecutea
commandinotherdirectories,includingthecurrentworking
directory,anexplicitpathmustbegiven
(/home/chris/demo_scriptor./demo_script,notjust
demo_script).
$PWD:Thisissetbytheshelltothepathnameofthecurrent
workingdirectory:
$cd$HOME&&echo$PWD
/home/chris
$cd"$puzzles"&&echo
$PWD
/data/cryptics
standard-vars—ACollectionofUsefulVariables
Mystandard-varsfilebeginswiththeselines:
NL='
'
CR='
'
TAB=''
Youmightbeabletoguessthatthesethreevariablesrepresent
newline,carriagereturn,andtab,butit’snotclear,andcannotbe
cutandpastedfromawebsiteornewsgroupposting.Oncethose
variablesaresuccessfullyassigned,however,theycanbeused,
withoutambiguity,torepresentthosecharacters.Thestandard-vars
fileisreadbytheshellandexecutedinthecurrentenvironment
(knownassourcing,itisdescribedlaterinthechapter)inmostof
myshellscripts,usuallyviastandard-funcs,whichappearslaterin
thischapter.
Createthefilewiththefollowingscript,thenaddothervariables
aswefoundthemuseful:
printf"%b\n"\
"NL=\"\n\""\
"CR=\"\r\""\
"TAB=\"\t\""\
"ESC=\"\e\""\
"SPC=\"\040\"\
"exportNLCRTABESCSPC">
$HOME/scripts/standard-vars-sh
The-shextensionispartofthesystemusedforworkingon
scriptswithoutcontaminatingtheirproductionversions.Itis
explainedinChapter20.
Patterns
Twotypesofpatternsareusedinshellscripts:pathnameexpansion
andregularexpressions.Path-nameexpansionisalsoknownas
globbing,andisdonebytheshell;regularexpressionsaremore
powerful(andmuchmorecomplicated),andareusedinexternal
commandssuchassed,awk,andgrep.
PathnameExpansion
Pathnameexpansionusesthreespecialcharacterstotelltheshellto
interpretanunquotedstringasapattern:
Theasterisk(*)matchesanystring,includinganemptyone.
Byitself,anasteriskmatchesallfilesinthecurrentdirectory,
exceptthosethatbeginwithadot.
Aquestionmark(?)matchesanysinglecharacter.Byitself,a
questionmarkmatchesallfilesinthecurrentdirectorywhose
nameisasinglecharacter,otherthanadot.
Anopeningbracket,([),whenmatchedwithaclosingbracket,
(]),matchesanyofthecharactersenclosed.Thesemaybe
individualcharacters,arangeofcharacters,oramixtureofthe
two.
Thesepatternscanbecombinedtoformcomplexpatternsfor
matchingstringsincasestatements,andforbuildinglistsoffiles.
Hereareafewexamplesexecutedinadirectorycontainingthese
files:
abc
deeef
egehfe
fffgfh
gegfgg
ghhehf
hghhi_158_d
i_261_ei_502_fi_532_b
i_661_ci_846_gi_942_a
j_114_bj_155_fj_248_e
j_326_dj_655_cj_723_g
j_925_ak_182_ak_271_c
k_286_ek_292_fk_294_g
Todisplayallfileswithsingle-characternames:
$echo?
abcd
Thenextexampleprintsallfileswhosenamesendwithf:
$echo*f
efffgfhfi_502_fj_155_fk_292_f
Allfilescontaininganumberwhosefirstdigitisintherange3
to6canbeshownwith:
$echo*_[3-6]*
i_502_fi_532_bi_661_cj_326_dj_
655_c
RegularExpressions
WhenIstartedwritingshellscripts,Ihadproblemswithgrep.I
usedtheasteriskasawildcard,expectingittomatchanystring.
Mostofthetime,allwaswell,butoccasionallygrepwouldprinta
lineIdidn’twant.Forinstance,whenIwantedlinesthatcontained
call,Imightgetcalculateaswell,becauseIused'call*'asthe
searchpattern.
Atsomepoint,itdawnedonmethatthepatternsusedbygrep
werenotthewildcardsIhadbeenusingforyearstomatchfiles,but
regularexpressions,inwhich*stoodfor“zeroormoreoccurrences
oftheprecedingcharacterorrangeofcharacters”.Tomatchany
string,thepatternis.*,astheperiodmatchesanycharacter,andthe
combinationmatches“zeroormoreoccurrencesofanycharacter.”
Aswithpathnameexpansion,[...]matchesanyofthecharacters
enclosedinthebrackets.
Tomatchnon-emptylines,searchforanysinglecharacter;that
is,adot:
$printf"%s\n"JanuaryFebruaryMarch""
MayJuneJuly|grep.
January
February
March
May
June
July
Toprintlinescontainingaborac,bracketsareused:
MayJuneJuly|grep'[bc]'
February
March
Inaddition,thecaret,^,matchestheexpressiononlyatthe
beginningofaline,andthedollarsign,$,matchesonlyattheendof
aline.Combiningthetwo,^...$,matchesonlytheentireline.By
anchoringthematchtothebeginningoftheline,wecanmatchlines
withaasthesecondletter(thefirstlettercanbeanything):
MayJuneJuly|grep'^.a'
January
March
May
Usingboththecaretandthedollarsign,wecanmatchlines
beginningwithJandendingwithy:
MayJuneJuly|grep'^J.*y'
January
July
Therearevariousflavorsofregularexpressions,includingbasic
(BREs)andextended(EREs).ThePerllanguagehasitsownset
(whichhasbeenincorporatedintoPython),butthebasicsare
commontoallversions.
Regularexpressionscanbeverycomplex(theexampleinthe
“Notes”totheprintatfunctioninChapter12isdauntingatfirst
glance,butactuallyfairlysimple),andaresometimesdescribedas
“writeonly”;oncearegex(orregexp,thecommonabbreviations
forregularexpression)iswritten,itcanbeveryhardtoreaditand
understandhowitworks.A.M.KuchlingputitwellinhisRegular
ExpressionHOWTO1(replacePythonwithwhateverlanguageyou
areusing):
Therearealsotasksthatcanbedonewithregularexpressions,
buttheexpressionsturnouttobeverycomplicated.Inthese
cases,youmaybebetteroffwritingPythoncodetodothe
processing;whilePythoncodewillbeslowerthanan
elaborateregularexpression,itwillalsoprobablybemore
understandable.
Ifyouwanttodelvedeeperintoregularexpressions,theclassic
bookfromO’Reilly,sed&awk,hasaverygoodsection,andthey
arecoveredcomprehensivelyintheApressbook,Regular
ExpressionRecipes:AProblem-SolutionApproach.Therearealso
somelinksintheAppendixtoonlineresources.Inthisbook,you
willfindveryfewregularexpressions,andnonethatcannotbe
easilyunderstood.
ParameterExpansion
Atitsmostbasic,parameterexpansionsubstitutesthevalueofthe
variablewhenitisprecededbyadollarsign($).Thevariablemay
beenclosedinbraces(${var}),andifthevariableisapositional
parametergreaterthan9,thebracesmustbeused.Youcanusethree
otherformsofexpansionwithinthebraces:Bourne,POSIX,and
shellspecific.
TheoriginalBourneshellparameterexpansionstestedwhether
thevariablewassetorempty,andactedontheresultsofthattest.
TheKornShelladdedexpansionstoreturnthelengthofthe
variable’scontents,andtoremovethebeginningorendofthevalue
ifitmatchedapattern;thesehavebeenincorporatedintothePOSIX
standard.KornShell93(ksh93)addedthesearchand-replaceand
substringcapabilitiesthathavealsobeenincludedinbash.
TheBourneShellExpansions
TheoriginalBourneshellexpansionshavetwoforms.Withacolon,
theytestwhetheravariableisnullorunset;withoutthecolon,the
testisonlywhetherthevariableisunset.
${var:-DEFAULT}
If$varisunsetornull,theexpressionexpandstoDEFAULT;
otherwise,itexpandstothecontentsofthevariable:
$var=
$echo${var:-y}
y
$var=x
$echo${var:-y}
x
Withoutthecolon,thevariablemustbeunset,notjustnull,for
DEFAULTtobeused(theresultofthevariableexpansionis
surroundedbyslashes):
$var=
$echo/${var-y}/
//
$unsetvar
$echo/${var-y}//y/
${var:=DEFAULT}
Theonlydifferencebetweenthisandthepreviousexpansionisthat
thisalsoassignsavaluetovar:
$var=
$echo"${var:=q}"
q
$echo"${var:=z}"
q
${var:+VALUE}
Thisexpansion(whichwasnotintheveryfirstBourneshell)isthe
oppositeoftheprevioustwo.Ifvarisnotnull(or,withoutthe
colon,ifitisset),VALUEisused.Inthefirstexample,varisunset,so
thevariableexpandstoanemptystring,withorwithoutthecolon:
$unsetvar
$echo/${var:+X}/
//
$echo/${var+X}/
//
Inthenextexample,varissetbutnull.Withthecolon,thetest
isforanon-nullstring,soXisnotprinted.Withoutit,Xisprinted,
becausethetestisforwhetherthevariableisset.
$var=
$echo/${var:+X}/
//
$echo/${var+X}//X/
Finally,whenthevariableissetandnotnull,VALUEisused,with
orwithoutthecolon:
$var=A
$echo/${var:+X}//X/
$echo/${var+X}//X/
Acommonuseforthistypeofexpansioniswhenbuildingalist
inwhichaseparatorcharacteriswantedbetweenitems.Ifwejust
usedconcatenation,we’dendupwiththeseparatoratthebeginning
whereitisnotwanted:
$forfinabcde
>do
>list=$list,$f
>done
$echo$list
,a,b,c,d,e
Withthisexpansion,wecaninserttheseparatoronlyif$listis
notempty:
list=${list:+$list,}$f
Thisisequivalenttothefollowing:
if[-n"$list"]
then
list=$list,$f
else
list=$f
fi
Usingthisexpansioninplaceofthesimplevariableinthe
precedingexample,thereisnoinitialcomma:
$forfinabcde
>do
>list=${list:+$list,},$f
>done
$echo$list
a,b,c,d,e
${var:?MESSAGE}
Ifvarisunset(or,withthecolon,null),anerrororMESSAGEwillbe
printed.Iftheshellisnotinteractive(asinthecaseofascript),it
willexit.
$unsetvar
$echo${var?}
bash:var:parameternullornotset
$echo${1?Novaluesupplied}
bash:1:Novaluesupplied
POSIXParameterExpansions
Theexpansionsintroducedbyksh,andadoptedbyPOSIX,perform
stringmanipulationsthatwereoncetheprovinceoftheexpr
command.Intheseexpansions,PATTERNisafile-globbingpattern,
notaregularexpression.
${#var}—LengthofVariable’sContents
Thisexpansionreturnsthelengthoftheexpandedvalueofthe
variable:
$var=LENGTH
$echo${#var}
6
${var%PATTERN}—RemovetheShortestMatch
fromtheEnd
Thevariableisexpanded,andtheshorteststringthatmatches
PATTERNisremovedfromtheendoftheexpandedvalue:
$var=usr/local/bin/crafty
$echo"${var%/*}"
usr/local/bin
${var%%PATTERN}—RemovetheLongest
MatchfromtheEnd
Thevariableisexpanded,andthelongeststringthatmatches
PATTERNfromtheendoftheexpandedvalueisremoved:
$echo"${var%%/*}"
usr
${var#PATTERN}—RemovetheShortestMatch
fromtheBeginning
Thevariableisexpanded,andtheshorteststringthatmatches
PATTERNisremovedfromthebeginningoftheexpandedvalue:
$echo"${var#*/}"
local/bin/crafty
${var##PATTERN}—RemovetheLongestMatch
fromtheBeginning
Thevariableisexpanded,andthelongeststringthatmatches
PATTERNisremovedfromthebeginningoftheexpandedvalue:
$echo"${var##*/}"
crafty
CombiningExpansions
TheresultofoneexpansioncanbeusedasthePATTERNinanother
expansiontoget,forexample,thefirstorlastcharacterofastring:
$var=abcdef
$echo${var%${var#?}}
a
$echo${var#${var%?}}
f
Shell-SpecificExpansions,bash2,andksh93
Iusetwoshell-specificparameterexpansionsinthisbook,eitherin
thebash/ksh93versionsoffunctions(forexample,substrin
Chapter3),orinbash-onlyscripts.
${var//PATTERN/STRING}—ReplaceAll
InstancesofPATTERNwithSTRING
Becausethequestionmarkmatchesanysinglecharacter,this
exampleconvertsallthecharacterstotildestouseasanunderline:
$var="Chapter1"
$printf"%s\n""$var""${var//?/
Chapter
1
}"
Thisexpansioncanalsobeusedwithasingleslash,which
meanstoreplaceonlythefirstinstanceofPATTERN.
${var:OFFSET:LENGTH}—ReturnaSubstring
of$var
Asubstringof$varstartingatOFFSETisreturned.IfLENGTHis
specified,thatnumberofcharactersissubstituted;otherwise,the
restofthestringisreturned.Thefirstcharacterisatoffset0:
$var=abcdefgh
$echo"${var:3:2}"
de
$echo"${var:3}"
defgh
ShellArithmetic
IntheBourneshell,allarithmetichadtobedonebyanexternal
command.Forintegerarithmetic,thiswasusuallyexpr.The
KornShellincorporatedintegerarithmeticintotheshellitself,andit
hasbeenincorporatedintothePOSIXstandard.Theformis$((
expression)),andthestandardarithmeticoperatorsaresupported:
+,-,*,/,and%,foraddition,subtraction,multiplication,division,
andmodulus(orremainder).Thereareotheroperators,buttheyare
notusedinthisbook;yourshell’sdocumentationwillhaveallthe
details.
Thestandardorderofoperatorprecedencethatweremember
fromhighschoolalgebraapplieshere;multiplicationanddivision
areperformedbeforeadditionandsubtraction,unlessthelatterare
groupedbyparentheses:
$a=3
$echo$(($a+4*12))
51
$echo$((($a+4)*12))
84
ThePOSIXspecificationallowsvariablesinarithmetic
expressionstobeusedwithoutaleadingdollarsign,likethis:echo
$((a+4))insteadofecho$(($a+4)).Thiswasnotclear
fromearlyversionsofthestandard,andamajorgroupofotherwise
POSIX-compliantshells(ash,dash,andshonBSDsystems)didnot
implementit.Inorderforthescriptsinthisbooktoworkinthose
shells,thedollarsignisalwaysused.
Aliases
Aliasesarethesimplereplacementofatypedcommandwith
another.InaPOSIXshell,theycanonlytakeargumentsafterthe
command.Theiruseinscriptsandonthecommandlinecanbe
replacedentirelybyfunctions;therearenoaliasesinthisbook.
SourcingaFile
Whenascriptisexecuted,itcanobtainthevaluesofvariablesthat
havebeenplacedintheenvironmentwithexport,butanychangesit
makestothoseorothervariableswillnotbevisibletothescriptthat
calledit.Functionsdefinedorchangestothecurrentdirectoryalso
willnotaffectthecallingenvironment.Forthesetoaffectthe
callingenvironment,thescriptmustbesourced.Byusingthedot
command,thefileisexecutedinthecurrentshell’senvironment:
.filename
Thistechniqueisusedthroughoutthebook,mostoftentodefine
alibraryoffunctions.
Functions
Functionsgrouponeormorecommandsunderasinglename.
Functionsarecalledinthesamewayasanyothercommand,
completewitharguments.Theydifferfromordinarycommandsin
thattheyareexecutedinthecurrentshellenvironment.Thismeans
theycanseeallthevariablesofthecallingshell;theydonotneedto
beexported.Variablessetorchangedinafunctionarealsosetor
changedinthecallingshell.And,mostofall,afunctionthatdoes
allitsworkwithshellcommandsandsyntaxisfasterthanan
externalcommand.
FunctionsAreFast
InChapter6,thebasenameanddirnamefunctionsreplacethe
externalcommandsofthesamename,anddothejobinafractionof
thetime.Evenafunctionmorethan70lineslongcanexecutemuch
fasterthananexternalcommand.InChapter5,the_fpmulfunction
isfasterthanthecalcfunction,whichusesawk,unlessthereare
dozensofoperands.
Undernormalcircumstances,onewouldn’tthinkofwritinga
shellfunctionforfloating-pointmultiplication;awkwouldbea
betterchoiceforthat.I(Chris)wrote_fpmulasapersonal
challenge,justtoshowthatitcouldbedone.Nowthatit’sdone,and
ithasprovedtobefasterthanothermethods,Idouseitinscripts.A
singlelineisallthat’sneededtomakethefunctionavailable:
.math-funcs
Otheroperationsondecimalfractionsaremorecomplicated,
andthereforearen’tworthwritingunlessthere’saspecificneedto
doso.
CommandSubstitutionIsSlow
WhenIdiscoveredthatusingcommandsubstitutiontostorethe
resultsofafunctioninavariablewassoslow(inallshellsexcept
ksh93)thatitseverelyreducedtheadvantageofusingfunctions,I
startedlookingforwaystomitigatethephenomenon.ForawhileI
triedusingavariabletotellafunctionwhethertoprinttheresult:
[${SILENT_FUNCS:-0}=1]||echo
"${_FPMUL}"
Thisworked,butIfoundituglyandcumbersome;whenIdidn’t
wantafunctiontoprintanything,IhadtosetSILENT_FUNCSto1
usuallybyprecedingthecallwithSILENT_FUNCS=1.Occasionally,I
couldsetitatthebeginningofasectionofcodeandhaveitinforce
forallsubsequentfunctioncalls.Iwaswellintowritingthisbook
whenthesolutionoccurredtome,andIhadtobacktrackand
rewritepartsofearlierchapterstoincorporateit.
Wheneverafunctionreturnsavalue(otherthananexitstatus),I
nowwritetwofunctions.Onehastheexpectedbehaviorofprinting
theresult;theother,whichbeginswithanunderscore,setsa
variablethatisthefunction’sname(includingtheunderscore)
convertedtouppercase.Toillustrate,hereisapairoffunctionsto
multiplytwointegers:
_mul()
{
_MUL=$(("$1"*"$2"))
}
mul()
{
_mul"$@"&&printf"%s\n""$_MUL"
}
Icannowprinttheresultofthemultiplicationwith
$mul1213
156
Or,Icanstoretheresultinavariablewith
$_mul1213
$product=$_MUL
Theextrafewmillisecondsittakestousecommandsubstitution
...
$timemul123456
56088
Real:0.000User:0.000System:0.000
$time{q=$(mul123456);}
Real:0.005User:0.001System:0.003
...maynotseemsignificant,butscriptsoftenloophundredsor
eventhousandsoftimes,andmayperformseveralsuch
substitutionsinsidealoop.Theresultisasluggishprogram.
UsingtheFunctionsinThisBook
Thefunctionsareusedinthreeways:atthecommandline,as
commandsinscripts,andasareference.Foruseatthecommand
line,thesourceofsomeofthefunctionlibrariesareintheshell
startupfile;othersareloadedatthecommandlinewhenneeded.In
scripts,usuallyasinglefunctionlibraryissourced,anditwillload
anyotherlibrariesitneeds.Atothertimes,thefunctionlibrary
servesasareference,andthecodeiscopied,sometimesmodifying
it,intothebodyofanotherscriptorfunction.Youmightwantto
followthisasyourstrategyorcouldhavesomethingcompletely
different.
Thefunctionsinthisbookaremostlystoredinlibrariesof
relatedfunctions.Youmayfindadifferentstructuremoresuitedto
yourcodingstyle.Ifso,goaheadandreorganizethem.Wewould
recommendthatyouavoidhavingthesamefunctioninmorethan
onelibrary,unlessthesubsequentversionsofferadditionalfeatures
thatarenotusuallyneeded.Thefirstlibraryinthisbookisa
collectionoffunctionsthatareusedinmanyscripts.Allareusedin
atleastonescriptinthebook.
standard-funcs:ACollectionofUseful
Commands
Thefunctionsinthislibraryencapsulatecommonlyusedtasksina
consistentinterfacethatmakesusingthemeasy.Whenyoumight
needtogetakeystrokefromtheuser,youcancallget_key;to
displayadate,useshow_date;wanttoexitfromascriptbecause
somethinghasfailed,usedie.Withmenu1,displayanythingfrom
one-linetofull-screenmenuandexecuteacommandbasedonthe
user’sresponse.
Afterdefiningthefunctions,thelibraryloadsstandard_vars
shownearlierinthischapter:
.standard-vars
1.1get_key—GetaSingleKeystrokefromthe
User
Insomecircumstances,suchaswhenaskingausertoselectfroma
menu,onlyasinglekeyneedstobepressed.Theshellread
commandrequiresanewlinebeforeitexits.Isthereawaytoreada
singlekey?
HowItWorks
Thebashreadcommandhasanoption,-n,toreadonlyaspecified
numberofcharacters,butthatislackinginmostothershells.The
portablewayusessttytoturnofftheterminal’sbuffering,andto
settheminimumnumberofcharactersrequiredforacompleteread.
Asinglecharactercanthenbereadbydd.
Usage
get_key[VAR]
Ifavariableisspecified,thekeywillbereadintothatvariable.
Ifnot,itwillbein$_KEY.
TheScript
get_key()
{
[-t0]&&{##Checkwhetherinputiscomingfrom
aterminal
[-z"$_STTY"]&&{
_STTY=$(stty-g)##Storethecurrent
settingsforlaterrestoration
}
##Bydefault,theminimumnumberofkeysthat
needstobeenteredis1
##Thiscanbechangedbysettingthedd_min
variable
##IftheTMOUTvariableissetgreaterthan0,
thetime-outissetto
##$TMOUTseconds
if[${TMOUT:--1}-ge0]
then
_TMOUT=$TMOUT
stty-echo-icanontime$(($_TMOUT*10
))min${dd_min:-1}
else
stty-echo-icanonmin${dd_min:-1}
fi
}
##Readakeyfromthekeyboard,
usingddwithablocksize(bs)of1.
##Aperiodisappended,orcommand
substitutionwillswallowanewline
_KEY=$(ddbs=1count=12>/dev/null;
echo.)
_KEY=${_KEY%?}##Removetheperiod
##Ifavariablehasbeengivenon
thecommandline,assigntheresulttoit
[-n"$1"]&&
##Duetoquoting,either'or"needsspecial
treatment;Ichose'
case$_KEYin
"'")eval"$1=\"'\"";;
*)eval"$1='$_KEY'";;
esac
[-t0]&&stty"$_STTY"##resetterminal
[-n"$_KEY"]##Succeedifakeyhasbeen
entered(nottimedout)
}
Notes
Theget_keyfunctionisoftenredefinedinotherscriptstoallow
entryofcursorandfunctionkeys—andevenmouseclicks.Foran
example,whichistoolongtoincludeinthisbook,seethe
mouse_demoscriptonmywebsite.2
1.2getline—PromptUsertoEnteraLine
Forinteractivescripts,Iliketheeditingcapabilitiesthatbash’s
readlinelibraryoffers,butIstillwantthescripttoworkinother
POSIXshells.Iwantthebestofbothworlds!
HowItWorks
Thegetlinefunctionchecksfortheexistenceofthe$BASH_VERSION
variable,andusesthereadlinelibraryifitisset.Ifnot,aPOSIX
readcommandisused.
Usage
getline"PROMPT"[VAR]
IfnoVARisgiven,thelineisreadintothe_GETLINEvariable.If
thevariablenameusedispassword,thekeystrokeswillnotbe
echoedtotheterminal.
TheScript
_getline()
{
##Checkthattheparametergivenisavalid
variablename
case$2in
[!a-zA-Z_]*|*[!a-zA-Z0-9_]*)die2"Invalid
variablename:$2";;
*)var=${2:-_GETLINE};;
esac
##Ifthevariablenameis"password"donotturnon
echoing
[-t0]&&["$2"!="password"]&&sttyecho
case${BASH_VERSION%%.*}in
[2-9]|[1-9][0-9])
read${TMOUT:+-t$TMOUT}-ep"$1:"-$var
;;
*)printf"%s:""$1">&2
IFS=read-r$var
;;
esac
[-t0]&&stty-echo##turnechoingbackoff
}
1.3press_any_key—PromptforaSingle
Keypress
Despitethenow-tritecomment,“Butmykeyboarddoesn’thavean
ANYkey,”itisoftendesirabletopauseexecutionofascriptuntil
theuserpressesakeywiththemessage“PRESSANYKEYTO
CONTINUE.”
HowItWorks
Theget_keyfunction(showntwofunctionspreviously)providesthe
mechanismtoreadasinglekeypress,andprintfandcarriage
returnsdisplayanderasethemessage.
Usage
press_any_key
Atonetime,thisscriptacceptedanargument:thenameofa
variableinwhichtostorethekey.Itwasneverused,soitwas
removed.Ifyoudowantthekeystroke,youcangetitfromthe
$_KEYvariablesetbyget_key.
TheScript
press_any_key()
{
printf"\r<PRESSANYKEY>"##Displaythe
message
get_key##Getthe
keystroke
printf"\r\r"##Erasethe
message
}
1.4menu1—PrintaMenuandExecutea
SelectedCommand
Overtheyears,Ihavewrittenmanymenuscripts,butwhenI
neededasimple,easilyadaptablemenu,nonefitthebill.AllI
wantedwasasimplescriptthatcoulddisplayamenu,andexecutea
commandbasedontheuser’sselection.
HowItWorks
Themenu1(socalledtopreventconflictwithanexistingmenu
program)functiondisplaysthe$menuvariable,whichshouldbeset
bythecallingscript.Thiscanbeanythingfromaone-linemenutoa
fullscreen.Theuserselectsbynumber,andthecorresponding
argumenttomenu1isexecutedwitheval.A0,q,oranewlineexits
themenu.
Usage
menu="MENU"menu1CMD1CMD2...
Twooptionalvariablescontrolthebehaviorofthefunction.If
$_MENU1isnotnull,thefunctionwillexitafterasinglesuccessful
loop.If$pause_afterisnotnull,thescriptwillpause,andtheuser
willbepromptedtopress_any_key.
Hereisasimplesession;thecharactersinboldaretheuser’s
input,whichdonotactuallyappearwhenthescriptisrun.
$menu="1.Yes2.No?"
$menu1"echoYES""echoNO"
1.Yes2.No?
3
bash:Invalidentry:3
1.Yes2.No?
2
NO
1.Yes2.No?
1
YES
1.Yes2.No?
q
Thisisaslightlyfancierversion,withatwo-linemenu,that
exitsafteronecommand:
$menu="$NLDoyouwanttosee
today’sdate?$NL1.Yes2.No?"
$menu1"date"":"
Doyouwanttoseetoday’sdate?
1.Yes2.No?y
bash:Invalidentry:y
Doyouwanttoseetoday’sdate?
1.Yes2.No?
MonFeb708:55:26EST2005
Formoreelaboratemenususingthisfunction,seethe
conversionscriptinChapter5.
TheScript
menu1()
{
m1_items=$###Numberofcommands(i.e.,
arguments)
while:##Endlessloop
do
printf"%s""$menu"##Displaymenu
get_keyQ##Getuser’schoice
case$Qin
0|q|""|"$NL")
printf"\n";break;;##
Breakout
[1-$m1_items])##
Validnumber
printf"\n"
(eval"\$$Q"
)##Executecommand
case$pause_after
in##Pauseifset
*?)press_any_key
;;
esac
;;
*)printf"\n\t\a%sInvalidentry:%s\n"
"${progname:+$progname:}""$Q"
continue
;;
esac
[-n"$_MENU1"]&&break##Ifset,exitafterone
successfulselection
done
}
1.5arg—PromptforRequiredArgumentIf
NoneSupplied
Youmayhaveanumberofscriptsthattakeanargument.You
mightwantasimplemethodtocheckwhetheranargumentis
present,andprompttheuserforoneifthereisn’t.
HowItWorks
Theargumentstothescriptarepassedusing"$@"toarg;ifthereis
anargument,itisstoredin$_arg;ifnot,theuserisprompted(with
thevalueof$prompt)toenteranappropriatevalue.
Usage
arg"$@"
ThebestillustrationofthisfunctioncomesfromChapter5,
wheretheconversionfunctionsalluseit.Here,forexample,isthe
scripttoconvertfromouncestograms(theactualcalculationis
doneby_oz2g):
oz2g()
{
units=grams
prompt=Ouncesarg"$@"
_oz2g$arg
printf"%s%s\n""$_OZ2G""${units:+$units}"
}
Whenrunwithoutanargument,argpromptsforinput.Ifthe
$unitsvariablehasbeensetbythecallingscript,itisprinted:
$oz2g
Ounces?
23
652.05grams
Andwithone,thereisnoprompt,andnounitsareprinted
becauseargempties$unitswhenthereisanargument:
$oz2g2
56.7
TheScript
arg()
{
case$1in
"")printf"%s?""$prompt">&2##Promptfor
input
sttyecho##Display
charactersentered
readarg</dev/tty##Get
user’sinput
;;
*)arg="$*"##Useexistingarguments
units=##Forusewiththe
conversionscript,Chapter5
;;
esac
}
1.6die—PrintErrorMessageandExitwith
ErrorStatus
Whenafatalerroroccurs,theusualactionistoprintamessageand
exitwithanerrorstatus.Itwouldbenicetomakethisexitroutine
uniformacrossallscripts.Thisissimilartowhat’savailableinPerl
orPHP.
HowItWorks
Thediefunctiontakesanerrornumberandamessageas
arguments.Thefirstargumentisstoredin$result,andtherestare
printedtothestandarderror.
Usage
dieNUM[MESSAGE...]
Thisfunctionisusuallyusedwhenanimportantcommandfails.
Forexample,ifascriptneedscertaindirectories,itmayuse
checkdirs(introducedlaterinthischapter)andcalldiewhenone
cannotbecreated:
checkdirs"$HOME/scripts""$HOME/bin"
||die13"Couldnotcreate$dir"
TheScript
die(){
result=$1
shift
printf"%s\n""$*">&2
exit$result
}
1.7show_date—DisplayDateinD[D]MMM
YYYYFormat
Muchofthetime,thedisplay_datefunctionfromthedate-funcs
libraryinChapter8isoverkill,andallyoumightwantisasimple
functiontoconvertadate,usuallyinISOformat,intoamore
humanreadableform.
HowItWorks
ByexpandingtheIFSvariabletoincludethehyphen,period,and
slash,aswellasaspace,mostpopulardateformatscanbesplitinto
theirseparatecomponents.Bydefault,theelementsareexpectedin
year-month-dayorder,but,bysettingtheDATE_FORMATvariableto
dmyormdy,theothercommonformatscanalsobeaccommodated.
Usage
_show_dateYYYY-MM-DD##Resultis
storedin$_SHOW_DATE
show_dateYYYY-MM-DD##Resultis
printed
Solongasyourememberthattheyear05istwomillenniainthe
past,youshouldhavelittletroublewiththisfunction.Herearea
fewexamples:
$show_date2005.04.01
1Apr2005
$DATE_FORMAT=dmyshow_date04-012005
4Jan2005
$DATE_FORMAT=mdyshow_date
04/01/2005
1Apr2005
TheScript
_show_date()
{
oldIFS=$IFS##SaveoldvalueofIFS
IFS="-./"##Allowsplittingonhyphen,period,
slashandspace
set--$*##Re-splitarguments
IFS=$oldIFS##RestoreoldIFS
##Iftherearelessthan3arguments,usetoday’s
date
[$#-ne3]&&{
date_vars##Populatedatevariables(see
thenextfunction)
_SHOW_DATE="$_DAY$MonthAbbrev$YEAR"
return
}
case$DATE_FORMATin
dmy)_y=$3##day-month-yearformat
_m=${2#0}
_d=${1#0}
;;
mdy)_y=$3##month-day-yearformat
_m=${1#0}
_d=${2#0}
;;
*)_y=$1##mostsensibleformat
_m=${2#0}
_d=${3#0}
;;
esac
##Translatenumberofmonthintoabbreviatedname
setJanFebMarAprMayJunJulAugSepOctNovDec
eval_m=\${$_m}
_SHOW_DATE="$_d$_m$_y"
}
show_date()
{
_show_date"$@"&&printf"%s\n""$_SHOW_DATE"
}
1.8date_vars—SetDateandTimeVariables
Youneedtosetvariableswithelementsofthecurrentdateand
time.It’spossibletodoitlikethis,butitisveryinefficient,andif
it’srunatthewronginstant(suchasjustbeforemidnight),the
resultswouldbeincorrectifthedatechangedbetweensetting,for
example,DAYandHOUR,because$HOURand$MINUTEwouldreferto
thedayafter$DAY:
YEAR=$(date+%Y)
MONTH=$(date+%m)
DAY=$(date+%d)
HOUR=$(date+%H)
MINUTE=$(date+%M)
SECOND=$(date+%S)
HowItWorks
Combiningtheshellcommand,eval,andthedateutility’sformat
string,itcanallbedoneinasinglecommand;dateiscalledwitha
formatstringthatproducesshellcodetosetallthevariables,and
evalexecutesit.
Usage
date_vars[DATEOPTIONS]
Theoutputofthedatecommandinthedate_varsfunction
lookslikethis:
DATE=2005-02-05
YEAR=2005
MONTH=02
DAY=05
TIME=22:26:04
HOUR=22
MINUTE=26
SECOND=04
datestamp=2005-02-05_22.26.04
DayOfWeek=Sat
DayOfYear=036
DayNum=6
MonthAbbrev=Feb
Thisisinterpretedasashellcommandbyevaltosetthe
variables.
Whiledate_varsisusuallycalledwithoutanyarguments,itcan
beusedwithwhateveroptionsyourdateutilityprovides.For
example,ifyouhaveGNUdate,youcanusethe-doptiontousea
dateotherthanthecurrentone:
date_vars-dyesterday
TheScript
date_vars()
{
eval$(date"$@""+DATE=%Y-%m-%d
YEAR=%Y
MONTH=%m
DAY=%d
TIME=%H:%M:%S
HOUR=%H
MINUTE=%M
SECOND=%S
datestamp=%Y-%m-%d_%H.%M.%S
DayOfWeek=%a
DayOfYear=%j
DayNum=%w
MonthAbbrev=%b")
##Removeleadingzeroesforuseinarithmetic
expressions
_MONTH=${MONTH#0}
_DAY=${DAY#0}
_HOUR=${HOUR#0}
_MINUTE=${MINUTE#0}
_SECOND=${SECOND#0}
##Sometimesthevariable,TODAY,ismore
appropriateinthecontextofa
##particularscript,soitiscreatedasasynonym
for$DATE
TODAY=$DATE
exportDATEYEARMONTHDAYTODAYTIMEHOURMINUTE
SECOND
exportdatestampMonthAbbrevDayOfWeekDayNum
}
1.9is_num—IsThisaPositiveInteger?
Whenascriptneedsaninteger,andthemethodofinputisnotunder
thescripter’scontrol(aswhenauserisaskedforinput),thevalue
shouldbeverified,andrejectedifitisnotwhatisrequired.
HowItWorks
Thisfunctionitselfislimitedtoverifyingpositiveintegers,and
checksforanynon-digitcharacters.Negativeintegerscanbe
acceptedbycallingthefunctionwithoutaleadingnegativesignon
theargument.
Usage
is_numINT
Theverificationisregisteredentirelybythefunction’sreturn
code:
$is_num33&&echoOK||echoNOGO
OK
$var=-33
$is_num"$var"&&echoOK||echoNO
GO
NOGO
Anegativeintegermaybeallowedbythecallerbystrippingany
leadingminussign:
$is_num"${var#-}"&&echoOK||
echoNOGO
OK
TheScript
is_num()
{
case$1in
*[!0-9]*)return5;;##Failisany
characterisnotadigitfrom0to9
esac
}
Notes
Ionceusedamoreelaborateversionthatwouldacceptnegative
integersaswellasaverboseoption,-v,whichwouldprintthe
results.Hereitis;usewhicheveryouprefer.
is_num(){
case${1#-}in
-v)isnum_verbose=1
shift
;;
*)isnum_verbose=0;;
esac
case$1in
*[!0-9]*)case$isnum_verbosein
1)printf"Notanumber:%s\n"
$1>&2;;
esac
return5
;;
esac
}
1.10abbrev_num—AbbreviateLargeNumbers
Whenprintingnumbersinaconfinedspace,suchasalistingof
mailboxcontentsinChapter10,printingtheminashortenedform
allowsmoreroomforothermatter.Youmightliketoconvert
123456to123K,and456789123to456MintoHumanreadableform.
HowItWorks
Thesuffix,K,M,orG,isdeterminedbythelengthofthenumber,and
thenumberisdividedby1000,1000000,or1000000000as
required.Theresultisalwaysamaximumoffourcharacters.
Usage
_abbrev_numNUM##Resultisstored
in$_ABBREV_NUM
abbrev_numNUM##Resultisprinted
Four-digitnumbersarenotconverted,astheyarenolongerthan
themaximum:
$abbrev_num4321
4321
Longernumberareconverted(androundedtothenearestunit):
$abbrev_num123456
123K
$abbrev_num234567890
235M
TheScript
Thenumbersareroundedupbyadding500,500000,or500000000
beforedividing.
_abbrev_num(){
case${#1}in
1|2|3|4)_ABBREV_NUM=$1;;
5|6)_ABBREV_NUM=$((($1+500)/1000))K;;
7|8|9)_ABBREV_NUM=$((($1+500000)/1000000))M;;
10|11|12)_ABBREV_NUM=$((($1+500000000)/
1000000000))G;;
*)_ABBREV_NUM="HUGE";;
esac
}
abbrev_num()
{
_abbrev_num"$@"&&printf"%s\n""$_ABBREV_NUM"
}
Notes
Theseabbreviatednumbersaresometimesreferredtoas“humanreadable,”buttheycanbelessusefulthanthefullnumberwhenit
comestocomparingrelativesizes.Theabbreviatedformcannotbe
sorted,andthelengthofanumbergivesagraphicrepresentationof
itssize.Thatrepresentationishelpedwhenthousandsseparatorsare
used,andforthatweneedafunction:commas.
1.11commas—AddThousandsSeparatorstoa
Number
Largenumbersareeasiertounderstandwhenthousandsseparators
areinserted.Canthatbedoneinascript?
HowItWorks
POSIXparameterexpansioncangivethelengthofavariableand
removethreedigits;this,plusconcatenatingthepartsinterspersed
withcommas,providesallthatisnecessary.
Usage
_commasNNN##Resultis
storedin$_COMMAS
commasNNN[NNN...]##Result[s]
is/areprinted
Theunderscoreversionofthefunctionacceptsonlyasingle
argumentandstoresitin_COMMAS,buttheotheracceptsmultiple
argumentsandprintstheresultsonetoaline.
$commas$((2345*43626))
3.14159265299792458
102,302,970
3.14159265
299,792,458
TheScript
_commas(){
_COMMAS=##Clearthevariableforthe
result
_DECPOINT=.##Characterfordecimalpoint;
adjustforotherlocales
_TH_SEP=,##Characterforseparator;
adjustforotherlocales
case$1in
"$_DECPOINT"*)_COMMAS=$1##Numberbeginswith
dot;noactionneeded
return
;;
*"$_DECPOINT")##Numberendswithdot;store
itin$c_decimal
c_num=${1%"$_DECPOINT"}
c_decimal=.
;;
*"$_DECPOINT"*)c_num=${1%"$_DECPOINT"*}##
Separateintegerandfraction
c_decimal=$_DECPOINT${1#*"$_DECPOINT"}
;;
*)c_num=$1##Nodot,thereforenodecimal
c_decimal=
;;
esac
while:
do
case$c_numin
##Three,twooronedigits[left]in$num;
##addthemtothefrontof_COMMASandexit
fromloop
???|??|?)
_COMMAS=${c_num}${_COMMAS:+"$_TH_SEP"$_COMMAS}
break
;;
*)##Morethanthreenumbersin$num
left=${c_num%???}##Allbutthelastthree
digits
##Prependthelastthreedigitsandacomma
_COMMAS=${c_num#${left}}${_COMMAS:+"$_TH_SEP"$_C
c_num=$left##Removelastthreedigits
;;
esac
done
##Replacedecimalfraction,ifany
_COMMAS=${_COMMAS}${c_decimal}
}
commas()
{
forn
do
_commas"$n"&&printf"%s\n""$_COMMAS"
done
}
1.12pr1—PrintArguments,OnetoaLine
Whenyouwanttopipealistofwordstoacommandoneatatime,
itmeansyoumustuseaforlooporpipetheitemstotr-s''
'\n'.Isthereabetterway?
HowItWorks
Iftherearemoreargumentsthanformatspecifiersinitsformat
string,printfwillreusethatstringuntilalltheargumentshavebeen
processed.Iftheformatstringcontainsoneformatspecifieranda
newline,eachargumentwillbeprintedonanewline.
Usage
pr1[-w]ITEM...
Withpr1,youcanprintalistofdirectoriesbeginningwiths,
onetoaline,withoutcallingls,andpipeitthroughawktogetthe
nameswithoutthefullpath.Thisisaccomplishedbysettingthe
fieldseparatortoaslash,andprintingthepenultimate(NF-1)field:
$pr1
$(NF-1)}'
/work/s*/|awk-F/'{print
screenshots
sounds
src
stocks
sun
sus
sysadmin
system
Evenusingtheexternalprintf,ifyourshelldoesnothaveit
builtin,pr1isconsiderablyfasterthanechoingthelisttols-1or
xargs-n1.
Bydefault,argumentswillbetruncatedtothewidthofthe
screen.Topreventthat,usethe-woption.
TheScript
pr1()
{
case$1in
-w)pr_w=;;
*)pr_w=-.${COLUMNS:-80};;
esac
printf"%${pr_w}s\n""$@"
}
1.13checkdirs—CheckforDirectories;CreateIf
Necessary
Ioftenneedtoknowwhetherallthedirectoriesascriptusesexist,
andcreatethemiftheydonot.
HowItWorks
Thecheckdirsfunctioncheckswhethereachofitsargumentsisan
existingdirectory,andattemptstocreateitifitdoesn’t;itprintsan
errormessageifadirectorydoesnotexistandcannotbecreated.If
anydirectorycouldnotbecreated,itreturnsanerrorstatus.
Usage
checkdirsDIRPATH...
Onlytheexistenceofadirectoryischeckedfor,nottheability
toreaditorwriteintoit.Inthisexample,/binexists,butitis
unlikelythatanordinaryuserhasthepermissiontowritetoafile
insideit.
$checkdirs/qwe/binuio/.autofsck
||echoFailed
mkdir:cannotcreatedirectory
`/qwe':Permissiondenied
mkdir:`/.autofsck'existsbutisnot
adirectory
Failed
TheScript
checkdirs(){
checkdirs=0##Returnstatus:successunlessacheck
fails
fordir##Loopthroughthedirectoryonthecommand
line
do
[-d"$dir"]&&##Checkforthedirectory
continue||##Ifitexists,proceedto
thenextone
mkdir-p"$dir"||##Attempttocreateit
checkdirs=1##Seterrorstatusif$dir
couldn’tbecreated
done
return$checkdirs##Returnerrorstatus
}
1.14checkfiles—CheckThataDirectory
ContainsCertainFiles
Withbashorksh93,youcanusebraceexpansiontocombine
multiplefilenameswithadirectoryandcheckfortheexistenceof
eachone:
$forfilein
/bin/{bash1,bash2,bash3,bash4,bash5}
>do
>[-f"$file"]||return5
>done
OtherPOSIXshellsdonothavethissyntax,soyouneedanother
methodofcombiningdirectoryandfilenameandcheckingthem.
HowItWorks
Thedirectorygivenasthefirstargumentisprependedtoeachofthe
otherargumentsinturn.Theresultingpathischecked,andthe
functionreturnswithanerrorassoonasagivenfiledoesnotexist.
Usage
checkfilesDIRFILE...
Thefunctionfailsifafiledoesnotexist,andthenameofthe
firstnonexistentfilewillbein$_CHECKFILE.
$checkfiles/binbash1bash2bash3
bash4bash5||
>printf"%s\n""$_CHECKFILEdoesnot
exist"
bash4doesnotexist
TheScript
checkfiles()
{
checkdict=$1##Directorymust
befirstargument
[-d"$checkdict"]||return13##Failif
directorydoesnotexist
shift##Removedir
frompositionalparameters
for_CHECKFILE##Loopthrough
filesoncommandline
do
[-f"$checkdict/$checkfile"]||return5
done
}
1.15zpad—PadaNumberwithLeadingZeroes
Whenaone-digitnumberneedstobezero-paddedtotwodigits,
suchaswhenbuildinganISOdate,it’seasytodowithparameter
expansion.Forexample,monthmaybeasingledigitortwodigits,
andmayormaynothavealeadingzero.Thisexpansionwillensure
itistwodigits,withaleading0ifnecessary:month=0${month#0}.
Whenmorethanonedigitmayhavetobeprependedandthe
number’slengthisvariable,amoreflexiblemethodisneeded.
HowItWorks
Theobviousmethodistouse$(printf"%0%${width}d""$NUM"),
butcommandsubstitutiontakesalongtimeinallshellsexcept
ksh93.Thebrute-forcemethodofaddingazeroinaloopuntilthe
desiredsizeisreachedisfasterunlesstheamountofpaddingis
large.Thedefinitionoflargemayvaryfromshelltoshelland
systemtosystem;onmysystemwithbashitisapproximately50.
Usage
_zpadNUMPADDING[CHAR]##Result
isstoredin$_ZPAD
zpadNUMPADDING[CHAR]##Result
isprinted
NUMisthenumbertobepadded;PADDINGisthedesirednumber
ofdigitsintheresult:
$zpad54
0005
TheoptionalCHARisacharactertouseforthepaddinginsteadof
0.
$zpad235x
xxx23
TheScript
_zpad()
{
_ZPAD=$1
while[${#_ZPAD}-lt$2]
do
_ZPAD=${3:-0}$_ZPAD
done
}
zpad()
{
_zpad"$@"&&printf"%s\n""$_ZPAD"
}
1.16cleanup—RemoveTemporaryFilesand
ResetTerminalonExit
Yourscriptsoftencoulddounfriendlythingstotheterminallike
changethecolors,changetheprompt,changesomesettings.These
areallrightsolongasthescriptisrunning,butwhencontrolreturns
tothecommandline,itmaybealmostunusable.Whenthescripts
exits,thisterminalneedstobereset.
Inaddition,thescriptsmaycreatetemporaryfilesthatshouldn’t
beleftlyingaroundwhenitisfinished.Theseneedtoberemoved.
HowItWorks
Thetrapcommandtellstheshelltoexecutecertaincommands
whenasignalisreceived.IfatrapissetontheEXITsignal,the
specifiedcommandswillbeexecutedwheneverthescriptfinishes,
whetherbyanormalcompletion,anderror,orbytheuserpressing
Control+C.Thecleanupfunctionisdesignedforthatpurpose.
Usage
trapcleanup[SIGNALS]
ThesignalwillusuallybeEXIT,andyoucouldhavethe
standard-funcslibrarysetthistrapwhenitissourced.
TheScript
cleanup()
{
##deletetemporarydirectoryandfiles
[-n"$tempfile_dir"]&&rm-rf"$tempfile_dir"
##Resettheterminalcharacteristics
[-t0]&&{
[-n"$_STTY"]&&stty$_STTY||sttysane
}
exit
}
trapcleanupEXIT##Settraptocall
cleanuponexit
TheUnixUtilities
Withafewexceptions,theexternalcommandsusedinthisbookare
standardonallUnixsystems.Hereweprovideaquicklookatmost
oftheutilitiesyou’llseeinthisbook.Formorecomplete
information,seetheirmanpagesorthePOSIXspecification(see
theAppendixforthewebsite,orChapter19forascripttoviewthe
POSIXmanpages).
cat:ConcatenateFilestotheStandardOutput
Thecommandcatisbestknownforandusedmoreoftenthannotto
displaythecontentsofafiletothescreen.Userscanoveruseitand
inmanycasesthecommandscanworkwithoutusingcat,some
examplescaninclude
cat"$1"|sed's/abc/ABC/g"
couldeasilybeusedsimplyas
sed'/s/abc/ABC/g'"$1"
or
cat/etc/passwd|grep"$USER"
couldbeusedas
grep"$USER"/etc/passwd
Inascript,validusesforcatincludethesetwotasks:
Toprovidemorethanonefilenameasinputtoacommandthat
canonlytakeasinglefileonthecommandline,orcanonly
readthestandardinput.Thisoftenappearsascat"$@"|CMD,
whichwillconcatenateanyfilesgivenasarguments,andread
thestandardinputifnofilesaregiven.
Tosendanunmodifiedfile,alongwiththeoutputofoneor
morecommands,totheinputofanothercommand.
sed:ATextStreamEditor
Manynewtoscriptingrarelyusesedformorethanrelativelysimple
tasks(iftheyuse).Itisaverypowerful,non-interactivetexteditor,
butitusescryptic,single-lettercommands.Thescripts,whichalso
usetheinherentlyhard-to-readregularexpressions,caneasily
becomedifficulttomaintain.
Onemainuseforsedisforsearchandreplace.Tochangeall
linesthatstartwith/binto/usr/bininthefilescript.shandstore
theresultinnewscript.sh,usethiscommand:
sed's|^/bin|/usr/bin|'script.sh>
newscript.sh
Theusualdelimiterafterthes(substitute)commandistheslash,
butanycharactercanbeused.Sincetheslashisinthestringsbeing
used,iteithermustbeescaped,oranothercharactermustbeused;
herethethepipesymbolisused.
Acommandinasedscriptcanbelimitedtocertainlinesor
certainrangesoflines.Todefinethisaddressspace,eithera
number(tomatchalinenumber)oraregularexpressionmaybe
used.Tolimitthechangestothefirsttenlines,andchangingall
instancesofJohntoJack:
sed'1,10s/John/Jack/g'
Tochangethefirstinstanceofoneonlyonlinescontaininga
numberofatleasttwodigits:sed'/[0-9][0-9]/s/one/1/'
Whennofileissuppliedonthecommandline,thestandard
inputisused.Alloutputissenttothestandardoutput.
awk:PatternScanningandProcessingLanguage
Theawkprogramminglanguagecandomanyofthesamethingsas
sed,butitusesamuchmorelegiblesyntax.Likesed,itprocessesa
filelinebyline,operatingonlinesthatmatchapattern.Whenno
patternisassociatedwithanaction,itisperformedonalllines;
whennoactionisassociatedwithapattern,thedefaultactionisto
printtheline.
Toprintonlynon-blanklines,aregularexpressionconsistingof
asingledotmatchesthoselineswhichcontainanycharacter(pr1,
fromthestandard-funcslibraryearlierinthischapter,printseach
argumentonaseparateline):
$pr1abc""ef|awk'/./'
a
b
c
e
f
Toprintthelastfieldlinesthatbeginwithanumber(NFisan
awkvariablethatcontainsthenumberoffieldsonthecurrentline;
thedollarsignspecifiesafieldontheline),usethis:
awk'/^[0-9]/{print$NF}'
Thefieldsare,bydefault,separatedbywhitespacecharacters.
Thefieldseparatorcanbechangedeitheronthecommandlinewith
the-FCHARoption,orinthescriptinaBEGINblock:
$awk'BEGIN{FS=":"}
/chris|cfaj/{print$1,$NF}'/etc/passwd
chris/bin/bash
cfaj/usr/local/bin/pdksh
grep:PrintLinesMatchingaRegularExpression
Likeawkandsed,grepreadsthestandardinputifnofilesare
specified,andcanacceptmultiplefilesonthecommandline.The
regularexpressioncanbeassimpleasastring(i.e.,aregular
expressionthatmatchesonlyitself):
$grep'^c'/etc/passwd
jayant:x:501:501:JayantC
Varma:/home/jayant:/bin/bash
chris:x:503:503:author:/home/chris:/usr/local/bin/pdksh
date:ShoworSettheSystemDate
Theflexibilityofdateisintheformatstring,whichdetermines
whatinformationaboutthecurrentdateandtimewillbeprinted,
andhowitwillbeprinted.Theformatstringbeginswitha+andcan
containliteralstringsandspecifiersfortheelementsofthedateand
time.Agoodexampleisinthedate_varsfunctioninthestandardfuncslibrary,shownpreviously.
Settingthesystemdatemustbedonebythesuperuser,and
doesn’tfigureinanyofthescriptsinthisbook.
tr:ACharacterTranslationUtility
Oftendevelopersaskwhytrcannotchangeonestringtoanother;
thereasonisthatitisacharactertranslationutility;usesedorawk
tochangestrings.Thetrutilityonlyconvertscharactersinthefirst
stringwiththecorrespondingcharactersinthesecondstring:
$echoabcdefgascdbf|trabc123
123defg1s3d2f
Withthe-doption,itdeletes(orratheromits)characters:
$echoabcdefgascdbf|tr-dabc
defgsdf
Seethemanpageforfulldetailsandotheroptions.
wc:CountCharacters,Words,andLinesinaFile
Bydefault,wcprintsthenumberoflines,words,andcharactersina
file,followedbythefilename.Theinformationprintedcanbe
selectedwiththe-l,-w,and-coptions.Ifanyoptionisgiven,
onlytheinformationrequestedisprinted:wc-lprintsonlythe
numberoflines,andwc-cwprintsthenumberofcharactersandthe
numberofwords.
Whennofilenameisgiven,wcreadsthestandardinput,and,
naturally,printsnofilename.Tostoreinformationfromwcina
variable,useinputredirectioninsteadofgivingafilename:
$wc-lFILE
3419FILE
$wc-l<FILE
3419
Someversionsofwcprintspacesbeforethesize.Thesecan
causeproblemswhenusingtheresultasanumberincalculation.
Therearevariouswaystoremovetheleadingspaces.Youcouldtry
usingshellarithmetic:
var=$(($(wc-l<FILE)))
file:DeterminetheFileType
Thefilecommanduses“magic”filesthatcontainrulesfor
classifyingfiles.
$file/etc/passwd/bin/ls./ean13.ps
/etc/passwd:ASCIItext
/bin/ls:ELF32-bitLSBexecutable,
Intel80386,version1(SYSV),forGNU/
Linux2.2.5,dynamicallylinked(uses
sharedlibs),stripped
./ean13.ps:ASCIIEnglishtext,withCRLF
lineterminators
ls:SortandProvideDetailsAboutFiles
Iflsiscalledwitharguments,itliststhefiles,orthecontentsof
directories,suppliedonthecommandline.Withoutanyarguments,
itliststhenamesoffilesanddirectoriesinthecurrentdirectory.
Thisexamplelistsfilesanddirectoriesinthecurrentdirectorythat
beginwithlettersintherangeatom;the-doptiontellsitjustto
printthenamesofdirectories,nottheircontents,and-Fprintsa
classificationcharacterafterthenamesofcertaintypesoffiles(in
thisexample,slashesindicatedirectories).
$l$ls-d-F[a-m]*
actionsean13.psfile2liblcms1_1.122ubuntu1_i386/
acuteenzyme.datfirefoxinstaller/max/
binenzymes.targlelogmc-chris/
ch8.docfile1lfy/menulog.
Tome,themostimportantoptionsare-l,whichprintsdetails
aboutthefilesinsteadofjustthenames,and-t,whichprintsthe
mostrecentlymodifiedfilesfirst.Manymoreoptionsareavailable;
asusual,seethemanpagefordetails.
uniq:RemoveConsecutiveDuplicateLines
Foruniqtoremoveallduplicatelinesfromafile,itmustfirstbe
sorted;otherwise,onlyconsecutiveduplicatelinesareremoved(see
uniqueinChapter19forascripttoremoveduplicatelinesfroman
unsortedfile).Sincethesortcommandhasa-uoptionthatremoves
duplicatelines,uniqismoreoftenusedwith-cforcountingthe
numberofinstancesofeachline(the{a..g}constructionwas
introducedinbash3,andexpandstoabcdefg):
$pr1$(($RANDOM%2)){a..g}$(($RANDOM
%2)){a..g}|sort|uniq-c
20a
10b
10c
20d
10g
11b
11c
21e
21f
11g
sudo:ExecuteCommandsastheSuperuser
Thefile,/etc/sudoers,containsrulesthatgiveuserspermissionto
runcommandsasotherusers,usuallyasroot.Permissioncanbe
giventoselecteduserstorunonlycertaincommandsorall
commands,withorwithoutapasswordbeingrequired,byprefacing
itwithsudo.
split:DivideaFileintoEqual-SizedPieces
Bydefault,splitdividesafileintoseparatefilesof1,000lines
each,usingxaa,xab,andsoonasthefilenameofthesplitfiles.The
usercanchoosetosplitafilebyanynumberofbytesornumberof
lines.Thenameandlengthoftheextensioncanalsobesetwith
command-lineoptions.With-lforlinecount,or-bforbytecount
andfileforthefilenameprefix.
which:ShowtheFullPathtoaCommand
Wehavegenerallyavoidedwhich,partlybecauseBourne-typeshells
havethebuilt-intypecommand,andpartlybecausehistoricallyit
wasacshscriptthatdidnotworkinaBourne-typeshell.Inrecent
years,however,whichisacompiledcommandonmostsystemsand
workswithallshells.Theadvantageithasovertypeisthatitprints
onlythepathtothecommand;type’soutputisnotstandard,and
usuallyincludesmorethanjustthepath.
gs,gv:Render,Convert,orViewPostScriptand
PDFFiles
GhostscriptisusedbyseveralcommandstorenderPostScript
programs;YoucouldusegvtoviewPSandPDFfiles,and
ImageMagick’sconverttoconvertfromoneformattoanother.
Summary
There’smoretotheshellthanisdescribedinthischapter.Itisonly
intendedtoexplainsomeofthecommandsandconceptsusedinthis
book’sscripts,notbeatutorialonhowtowriteshellscripts.
Throughoutthebook,thescriptsarecommentedtodescribewhatis
goingon,buttheydonotalwaysexplainthesyntaxitself.This
chapterhasprovidedthoseexplanations.
Footnotes
1 http://www.amk.ca/python/howto/regex/regex.html
2 http://cfaj.freeshell.org/src/scripts/mouse-demo-sh
Chapter2:PlayingwithFiles:
Viewing,Manipulating,andEditing
TextFiles
(1) Ontario,Canada
TextfilesareamainstayofUnixoperatingsystems.Fromthe
passwordfile,/etc/passwd,toserverconfigurationfiles,suchas
httpd.conf(exactlocationvaries,dependingonthesystem),a
computer’sbehavioriscontrolledbytextfiles.Butwhatisatext
file?E-mailandUsenetmessagesaretextfiles;HTMLwebpages
aretextfiles;shellscriptsaretextfiles;logfilesaretextfiles.Text
filesarehuman-readableandcanbeeditedwithanytexteditor.
Theycan,ofcourse,beviewed,manipulated,andeditedwithshell
scripts.
AUnixoperatingsystemdoesnotdistinguishbetweentextfiles
andbinaryfiles.Allfilesarebinaryfiles—textfilesarejusta
subset.AccordingtotheSingleUNIXSpecification(SUS)glossary,
atextfile
containscharactersorganisedintooneormorelines.Thelines
mustnotcontainNULcharactersandnonecanexceed{
LINE_MAX}bytesinlength,includingthenewlinecharacter1
NULisASCII0,abyteinwhichallbitsaresetto0,and
LINE_MAX,themaximumnumberofcharacterspermissibleonaline,
issystem-dependent;itis2,048ontheLinuxandFreeBSDsystems
Iuse.
ManyofthetextfilesonaUnixsystemuseawell-defined
format.Someofthesearecoveredinothersectionsofthebook,as
arescriptstoconverttexttootherformats.Thischapterpresents
scriptsthatdealwitharbitrarilyformatted(orunformatted)text
files.
EndoftheLineforOSDifferences:Converting
TextFiles
Filesareoftenmovedfromoneoperatingsystemtoanother.It’sa
loteasiernowthanitusedtobe.Gonearethedayswheneach
computermanufacturerusedadifferentcharacterencoding.The8bitCommodorecomputersIownedinthedim,distantpasthadthe
upper-andlowercasecharactersreversed(well,sortof,butthat’s
anotherstory).IBMmainframecomputersusedtheExtended
Binary-CodedDecimalInterchangeCode(EBCDIC)characterset.
Nowadays,theAmericanStandardCodeforInformation
Interchange(ASCII)isthebasisforallcoding.Thereareanumber
ofextensions,butthefirst128charactersareimmutable.Almost.
Therearedifferencesbetweenthenewlinecharacterson
differentsystems.
UnixsystemsuseNL(thenewlinecharacter,whichisASCII10,
octal\012,hexadecimal0x0a,alsocalledalinefeed,LF)toend
aline.
WindowsusestheCR/LFpair(CRisacarriagereturn,ASCII13,
octal\015,hexadecimal0x0d,ortousetheformmostoften
seen,^M).
MacintoshusesCR.
Whilemanyapplicationscandealwithdifferentlineendings,
therearemany,probablymore,thatcannot.Mostsystemsalready
havecommandstotakecareofit,dos2unixandunix2dosbeingthe
mostcommon,alongwiththeGNUcommandrecode.Ifyoudon’t
havethosecommands,thereplacementsforthemareprovidedin
thissection,alongwithconvertersforMacintoshfiles.
Inallthescriptsinthissection,theresultisstoredina
temporaryfilecreatedbymktemp(ifyoudon’thavethecommand,
there’sascripttoreplaceit),thenmvedovertheoriginalfile.
2.1dos2unix—ConvertWindowsTextFilesto
Unix
MS-DOSandWindowssystemsuseaCR/LFcombinationattheend
ofeachline.UnixsystemsuseonlyLF.Dependingonhowfilesare
copiedfromonesystemtoanother,thelineendingsmayormaynot
beconverted.AscriptcopiedfromWindowscouldlooklikethisin
atexteditor:
#!/bin/sh^M
echo"Defenestrationrequired"^M
exit^M
Manyprogramswillcomplainifafilehasincorrectline
endings.Somewilljustnotwork.Ashellscriptwithcarriage
returnsmayfailattheveryfirstline,withanerrormessage:
:badinterpreter:Nosuchfileor
directory
Ifthere’sacarriagereturnattheendoftheshebangline(e.g.,#!
/bin/sh^M),theoperatingsystemwilllookforaninterpreterwhose
nameendswithaCR,andwill(almostcertainly)notfindone.The
scriptwillnotrun.
HowItWorks
Thisscriptusessedtoremovethecarriagereturns.Onastring,ora
fewlines,youcouldremovetheCRsintheshellitself
(line=${line%$CR}),butsedisfasteronallbutveryshortfiles.
Carriagereturnsarelegitimatecharactersintextfiles,evenin
shellscripts.Insomescripts,aliteralCRisassigntoavariable,so
thatitisnotremovedbyaconversionprogram.Thisscriptremoves
carriagereturnsfromlineendingsbutleavesthemanywhereelseon
aline.
Usage
dos2unixFILE[...]
TheScript
CR=$(printf"\r")##orsource
standard-vars
progname=${0##*/}
tempfile=$(mktemp$progname.XXXXXX)
||exit5
forfile
do
sed"s/$CR$//""$file">"$tempfile"&&
mv"$tempfile""$file"
done
2.2unix2dos—ConvertaUnixFiletoWindows
UnderWindows,afilewithUnixlineendings(justLF),maylook
evenstrangerthanaWindowsfileonUnix:
#!/bin/sh
echo"Defenestrationrequired"
exit
Tip
Whilesomemethodsofcopyingfromonesystemtoanotherwill
dotheconversion,itcouldbesafertoconvertafileexplicitly
beforecopying.
HowItWorks
Thisscriptaddsacarriagereturntotheendofeachlinesothatall
lineshaveaCR/LFpair.ExistingCRswillnotbeduplicated.
Usage
unix2dosFILE[...]
TheScript
progname=${0##*/}
CR=$(printf"\r")
||exit5
forfile
do
sed-e"s/$/$CR/"-e"s/$CR$CR$/$CR/""$file">
"$tempfile"&&
done
2.3mac2unix—ConvertMacintoshFilestoUnix
Macintoshfilesuseacarriagereturninsteadofalinefeedastheline
terminator.ToaUnixsystem,thesefilesappearasasinglelong
line.Sincetheyhavenolinefeedcharacters,theyarenotvalidUnix
textfiles.
OnMacOS/X,whichisUnix-based,youmayneedtodo
conversionsevenifyou’renotcopyingtoanothersystem.Shell
scriptsneedUnixlineendings,andmanyUnixtoolsonOS/X
requireLF,notCR,betweenlines.
HowItWorks
MacfilescanpresentevenmoreproblemsthanWindowsfiles.
Sincetherearenolinefeeds,theentirefileisonasingleline.Many
Unixtextutilitieswillnotoperateonlonglines.
Thisscriptusestr,whichhasnosuchlimitationssinceitreads
andtranslatesindividualcharacters.
Usage
mac2unixFILE[...]
TheScript
progname=${0##*/}
tempfile=`mktemp$progname.XXXXXX`||
exit5
forfile
do
tr'\015''\012'<"$file">"$tempfile"&&
done
2.4unix2mac—ConvertUnixFilestoMac
Format
MacOS/XsometimesrequiresUnix-typelineendings.Insuch
cases,nochangestotheUnixlineendingsarenecessary.Forthe
othertimes(andforpre-OS/Xsystems),conversionmustbedone.
HowItWorks
Sincethisconversionisasimpleone-to-onecharactertranslation,tr
istheobvioustooltouse.Thisscriptconvertslinefeedstocarriage
returns,i.e.,fromUnixtoMacformat.
Usage
unix2macFILE[...]
TheScript
progname=${0##*/}
||exit5
forfile
do
##Oldversionsoftrdonotunderstand\rand\n
##theoctalcodesworkonallversions
tr'\012''\015'<"$file">"$tempfile"&&
done
2.5dos2mac—ConvertWindowsFilesto
Macintosh
ToconvertMS-DOS/WindowsfilestoMacintoshformat,theLF
mustberemoved.
HowItWorks
Alazywaycouldbetocombinetwoexistingscriptslike:
dos2unix"$@"
unix2mac"$@"
Itismoreefficient,however,justtoremovethelinefeedswith
tr.
Usage
dos2macFILE[...]
TheScript
progname=${0##*/}
||exit5
forfile
do
tr-d'\012'<"$file">"$tempfile"&&
done
2.6mac2dos—ConvertMacintoshFilesto
Windows
ToprepareaMacfileforuseonWindows,thecarriagereturnsmust
beconvertedtoCR/LFpairs.
HowItWorks
Theobvioustoolissed,butsomeversionsarelimitedinthe
maximumlinelengththeycanprocess,andanentireMacfilelooks
likeasingleline.Togetaroundthis,mac2dosfirstusestrtoconvert
theCRstoLFs.
Usage
mac2dosFILE[...]
TheScript
progname=${0##*/}
exit5
forfile
do
{tr'\015''\012'<"$file"||continue;}|
sed-e"s/$/$CR/">"$tempfile"&&
done
DisplayingFiles
Therearemanywaystodisplayafile.Thesimplestiscat,which
copiesafileorfilestostdout.ThePOSIXspecificationforcatonly
requiresoneoption,-u,whichmaybeignored.
Twocommonpagers,moreandless,enableausertoscroll
throughthefileonescreenatatime.Toprintafilewithpagebreaks
ormultiplecolumns,there’spr.Thefollowingscriptsdisplaythe
contentsoffilesindifferentways.
2.7prn—PrintFilewithLineNumbers
“Iwanttonumberthelinesofafile.”
“SodoI,butIdon’twantblanklinesnumbered.”
“Iwanttheblanklinesnumbered,butIwantconsecutiveblank
linestobesuppressed;Ijustwantoneblanklinewhenthereare
more,oneafteranother,inthefile.”
HowItWorks
Manyversionsofcatcandoallthat,butsomecannot,andline
numberingisnotrequiredbythePOSIXstandardforcat.Thereare
othercommandsthatcanprintafilewithlinenumbers:lessandpr.
Notallversionsofprsupportlinenumbering,thoughPOSIX
requiresit.
Thisscriptaddsoptionsthatarepresentinsomeversionsofcat:
squeezeadjacentblanklines;donotnumberblanklines;start
numberinganumberotherthan1.Andonethatmightnotbeseen
elsewhere:truncatelinesthatexceedtheterminalwidth.
Usage
prn[-b][-s][-nN][-t][FILE...]
Ifnofileisgivenonthecommandline,standardinputwillbe
used.Therearefiveoptions:
b:Donotnumberblanklines.
s:Squeezemultipleconsecutiveblanklinesintoone.
nNUM:StartnumberingwithNUM.
NNUM:NumberingwillbeNUMcharacterswide.
-t:Truncatethelinestothewidthofthescreenorwindow.
$printf"%s\n""the""quick""brown"
"fox""""jumped""over"|prn-b
1the
2quick
3brown
4fox
5jumped
6over
$prn-tN2kjv.txt|head
1Genesis:01:001:001:Inthebeginning
Godcreatedtheheavenandtheearth.
2Genesis:01:001:002:Andtheearth
waswithoutform,andvoid;anddarknesswas
3Genesis:01:001:003:AndGodsaid,
Lettherebelight:andtherewaslight.
4Genesis:01:001:004:AndGodsawthe
light,thatitwasgood:andGoddividedt
5Genesis:01:001:005:AndGodcalled
thelightDay,andthedarknesshecalledN
6Genesis:01:001:006:AndGodsaid,
Lettherebeafirmamentinthemidstofthe
7Genesis:01:001:007:AndGodmadethe
firmament,anddividedthewaterswhichw
8Genesis:01:001:008:AndGodcalled
thefirmamentHeaven.Andtheeveningandt
TheScript
progname=${0##*/}
version=1.0
die(){
error=$1
shift
printf"%s:%s\n""$progname""$*">&2
exit$error
}
blank=1
truncate=0
suppress=0
nwidth=5
cchar=
last=
opts=bstn:N:
n=1
whilegetopts$optsvar
do
case$varin
b)blank=0;;
s)suppress=1;;
n)case$OPTARGin
*[!0-9]*)die5"Numericvaluerequired"
esac
n=$OPTARG
;;
N)case$OPTARGin
*[!0-9]*)die5"Numericvaluerequired"
esac
nwidth=$OPTARG
;;
t)truncate=1;;
esac
done
shift$(($OPTIND-1))
if[$truncate-eq1]
then
width=$(($COLUMNS-${nwidth}+1))
fmt="%${nwidth}d%-${width}.${width}s\n"
else
fmt="%5d%s\n"
fi
cat${1+"$@"}|{
whileIFS=read-rline
do
case$linein
"")["$last"=""]&&[$suppress-eq1]&&
continue
[$blank-eq0]&&{
echo
last=$line
continue
};;
esac
printf"$fmt"$n"$line"last=$line
n=$(($n+1))
done
}
2.8prw—PrintOneWordperLine
Forvariousreasons,youmightsometimesneedtoprintallwordsin
afileonseparatelines.Someofthosereasonsareshowninother
scriptsinthisbook(wfreq,forexample,whichappearslaterinthe
chapter).
HowItWorks
AsoftenhappenswhenIcommitarelativelysimpletasktoafile,I
thinkofembellishments.Originallydesignedsolelytopipeafile,
onewordatatime,tootherscripts,prwhasbeensomewhat
enhanced.Onecannowspecifymorethanonewordperline,and
specifythewidthofthecolumnusedforeachword.Thewordscan
beprintedflushleftorflushrightwithinthatwidth.
Inputcanbeoneormorefiles,orthestandardinputifnofileis
givenonthecommandline.
Usage
prw[OPTIONS][FILE..]
-nNUM:PrintNUMwordsperline.
-wNUM:UseNUMcolumnsperword.
-WNUM:PrintwordsflushrightinNUM
columns.
Hereareafewexamples:
$prw$HOME/.bashrc|head-8
#
/.bashrc:
executed
by
bash(1)
for
non-login
shells.
$echo"thequickbrownfoxjumps
overthelazydog"|prw-n2-w20
thequick
brownfox
jumpsover
thelazy
dog
$echo"Nowisthetimeforallgood
mentocometotheaidoftheparty"|
prw-n3-W15
Nowisthe
timeforall
goodmento
cometothe
aidofthe
party
TheScript
die(){
printf"%s:%s\n""$progname""$*">&2
exit$1
}
NL=``
version="1.0"
progname=${0##*/}
num=0
flush_r=
err1="Numericvaluerequired"
whilegetoptsVn:w:W:var
do
case$varin
n)case$OPTARGin
*[!0-9]*)die"$err1";;
esac
num=$OPTARG
;;
w)case$OPTARGin
*[!0-9]*)die1"$err1";;
esac
width=$OPTARG;;
W)case$OPTARGin
*[!0-9]*)die1"$err1";;
esac
width=$OPTARG
flush_r=;;
V)printf"%s:%s\n""$progname""$version";exit
;;
esac
done
shift$(($OPTIND-1))
fmt=%${width:+$flush_r$width.$width}s
n=0
while[$((n+=1))-lt$num]
do
fmt="${fmt:+$fmt}%${width:+$flush_r$width.$width}s"
done
cat${1+"$@"}|
whileread-rline
do
printf"${fmt:-%s}\n"${line}
done
2.9wbl—SortWordsbyLength
Bytheirverynature,shellscriptsareoftenspecializedforvery
esotericpurposes.Therearemanysuchscripts.Whoelsewould
wanttosortalistofwordsbytheirlengths?
Infact,someoneinthecomp.unix.shellnewsgroupaskedhow
tosorttheoutputoflsbylengthoffilename.Whydidtheywant
that?noidea;perhapsitwasahomeworkexercise.Buttherewasa
scriptlyingaroundthatdidthejob.
Myreasonforhavingitrelatestooneofmyprofessions(Chris):
Icomposecrypticcrosswordsforamagazineandanewspaper.
Havingwordssortedbylengthisanabsolutenecessityforfinding
wordstofitinacrosswordgrid.Italsoaccountsforthenumberof
scriptsinthisbookthatplaywithwords(andcanbeusedtohelp
solvecrosswordpuzzlesortosetthem).
HowItWorks
Byprintingthewordsoneeach,flushrightonverywidelines,sort
will,ineffect,sortbywordlength.Theleadingspacesarethen
trimmed.Theoriginalscriptwasaone-liner,butsomeoptionshave
beenadded.
Usage
wbl[-ufr][-RRANGE][FILE...]
Allofthefouroptionsmaybeusedatonce:
-u:Eliminateduplicates.
-f:Makethesortcaseinsensitive(Aequalsa).
-r:Reversethesortingorder,printlongestwordsfirst.
-R"RANGE":AcceptallcharactersinRANGEaspartofaword.
Thedefaultis“[A-Z][a-z]”;toincludehyphenatedwordsasa
singleword,use“[A-Z][a-z]–”.Notethatahyphenmustnot
appearatthebeginningofRANGE,norbetweentwo
charactersthatarenottheboundariesofarange.
$printf"%s\n"thequickbrownfox
jumpsoverthelazydog|wbl-u
dog
fox
the
lazy
over
brown
jumps
quick
TheScript
range=[a-z][A-Z]
opts=
whilegetoptsufrR:var
do
case$varin
u)opts="$opts-u";;
f)opts="$opts-f";;
r)opts="$opts-r";;
R)range=$OPTARG;;
esac
done
shift$(($OPTIND-1))
##convertanycharacterthatisnot
alettertoanewline
out=$(cat${1+"$@"}|tr-cs"$range"
"[\012*]")
##printallthewordsflushright
across99columns
out=$(printf"%99s\n"$out|sort
$opts)
##printallthewordswithoutthe
leadingspaces
printf"%s\n"$out
FormattingFileFacts
Onecanusemanycommandstogetinformationaboutafile.Some
areuniversal;somearelimitedtocertainoperatingsystems.The
nicest(Ihavecomeacross)isGNUstat,whichisstandardon
recentGNU/Linuxsystems.Itallowsyoutoselecttheinformation
youwantandtheformatoftheoutput.Othersincludels,wc,and
file.
Thescriptsinthissectioncombinethesecommandsandothers
togiveyouevenmoreinformationaboutafileorfiles.
2.10finfo—FileInformationDisplay
Oneoftenwantsmoreinformationaboutafilethananysingle
utilityprovides,butcombiningtheoutputofmultiplecommands
canbetricky.
HowItWorks
Muchoftheinformationaboutafilecanbegleanedfromtheoutput
ofls-l.Thefinfocommandparsestheoutputofthatcommand,
thenuseswcandfiletogetmoreinformationaboutthefile.
Usage
finfo[OPTIONS]FILE[...]
Theoptionsareasfollows:
-z:Donotshowfilesofzerolength.
-D:Donotshowdirectories.
-H:Donotprintheader(insummarymode).
-s:Showsummary,onelineperfile.
-c:Clearscreenbetweenfiles(expandedmode).
-h:Printusageinformationandexit.
-V:Printscript’sversionandexit.
Ifyouwantasummaryofyourfiles,usefinfo-s:
LINESWORDSSIZEFILE
NAMEFILETYPE
271851265Ack.shtmlHTMLdocumenttext
1076528bio.txtASCIIEnglishtext
1052842148cpsh-shashscripttext
executable
631331164dehex-shBourne-Againshell
scripttextexecutable
1646330dos2mac-shashscripttext
executable
Ifyouwanteachfile’sinformationinaseparatesection,use
finfowithoutanyoptions(exceptthefilepattern),forexample:
$finfoAck.shtml
Name:Ack.shtml
Type:HTMLdocumenttext
Inode:611116
Size:1265
Lines:27
Words:185
Owner:group:chris:chris
Permissions:-rw-rw-r-
Date:Aug1215:51
TheScript
file_info()
{
ifile=$1
[-r"$ifile"]||return5
[-d"$ifile"]&&[$nodirs-ge1]&&return1
set--`wc"$file"2>/dev/null``file
"$file"`
lines=$1
words=$2
size=$3
shift5
file_type=$*
case$filein
*[-.]sh)desc=`grep-i"description=""$file"`
desc=${desc#*description=}
desc=${desc%%$NL*}
;;
*)desc=;;
esac
}
NL=
progname=${0##*/}
version=1.0
cu_up="\e[A"##ANSIcodetomove
cursoruponeline
summary=0
clear=0
noheader=0
empty=0
nodirs=0
opts=csHVz
do
case$varin
c)clear=1;;
D)nodirs=1;;
H)noheader=1;;
s)summary=1;;
z)empty=1;;
V)printf"%s\n""$progname,version$version";
exit;;
esac
done
shift$(($OPTIND-1))
_ll=6
_wl=6
_cl=7
_fl=25
_tl=$(($COLUMNS-$_ll-$_wl-$_cl
-$_fl-4))
_sfl=$(($COLUMNS-15))
##createabarwideenoughtofill
thewidthofthescreenorwindow
bar=---------------------------------
--------
while[${#bar}-lt${COLUMNS}]
do
bar=$bar$bar$bar
done
##oryoucanusetherepeatfunction
fromthenextchapter
##bar=$(repeat---$COLUMNS)
[$noheader-eq0-a$summary-eq1]
&&{
printf"\n%${_ll}s%${_wl}s%${_cl}s%-${_fl}.${_fl}s
%-${_tl}.${_tl}s\n"\
LINESWORDSSIZE"FILENAME""FILETYPE"
printf"%${COLUMNS}.${COLUMNS}s""$bar"
}
##############################################################
##Ifyouareusinganolderversion
ofSolarisorSunOS,
##inwhichls-ldoesn’tprintthe
group,uncommentthefollowingline
##ls_opt=-g
##############################################################
ls-il$ls_opt"$@"|
whilereadinodepermslinksowner
groupsizemonthdaytimefile
do
[$empty-eq1-a${size:-0}-eq0]&&continue
##Extractfilenameifit’sasymboliclink;
##thiswillfailintheunlikelyeventthat
##thetargetofthelinkcontains"->"
##SeeChapter6forascripttofix
badlyformedfilenames
case$permsin
l*)file=${file%-\>*};;
esac
file_info"$file"||continue
if[$summary-ge1]
then
printf"%${_ll}d%${_wl}d%${_cl}d%-${_fl}.${_fl}s
%-${_tl}.${_tl}s\n"\
$lines$words$size"$file""$file_type"
else
[$clear-ge1]&&printf"${CLS:=`clear`}"
printf"\n\n"
printf"Name:%-${_sfl}.${_sfl}s\n""$file"
printf"Type:%-${_sfl}.${_sfl}s\n"
"$file_type"
printf"Inode:%-${_sfl}.${_sfl}s\n"
"$inode"
[-n"$desc"]&&
printf"Description:%-${_sfl}.${_sfl}s\n""$desc"
printf"Size:%-${_sfl}.${_sfl}s\n""$size"
printf"Lines:%-${_sfl}.${_sfl}s\n"
"$lines"
printf"Words:%-${_sfl}.${_sfl}s\n"
"$words"
printf"Owner:group:%-${_sfl}.${_sfl}s\n"
"$owner:$group"
printf"Permissions:%-${_sfl}.${_sfl}s\n"
"$perms"
printf"Date:%-${_sfl}.${_sfl}s\n""$month
$day$time"
printf"\n\nPressENTER(q=Quit)\r"
readx</dev/tty
case$xin
q|Q)break;;
esac
printf"$cu_up%${COLUMNS}.${COLUMNS}s\n""$bar"
fi
done
2.11wfreq—WordFrequency
Whatisthemostfrequentlyusedwordinthefile?HowcanIgeta
listofallthewordsandthenumberoftimestheyappear?
HowItWorks
Usewfreqtogetalistofallthewordsandtheirfrequencyinoneor
morefiles.Ifnofileisgivenonthecommandline,standardinputis
counted.
Usage
wfreq[FILE...]
TheScript
prw${1+"$@"}|
tr-cd'a-zA-Z-\n'|
sort|
uniq-c|
sort-n
2.12lfreq—LetterFrequency
Shellscriptscanbegoodforprovidinginterestingtidbitsof
information,likeprintingthenumberofinstancesofeachletterof
thealphabetinapieceoftext.
HowItWorks
Whenafriendaskedwhetheritwastruethatcertainlettersoccurred
inmycrosswordpuzzlesmorefrequentlythanwouldbeexpectedin
normaltext,Iwrotethefirstversionofthisscript.Afterrunninga
fewpuzzlesthroughit,Iwassomewhatsurprisedthat,despitethe
smallsample(150to160lettersperpuzzle),therangewas
consistentwithtypicaldistributionintheEnglishlanguage.
Usage
lfreq[FILE...]
Asamplecrypticcrosswordgridshowsafairlynormal
distributionoflettersfortheEnglishlanguage:
$grid='
SOMEBODY.ABUSES
Y.Y.A.I...E.E.I
PROGRESS.TRIPOD
H.P.T.C...E.A.E
OXIDE.OVERTHROW
N.A.N.N.D...A.A
....DETRIMENTAL
T.T.E.I.F.N.E.K
REARRANGING....
O.P...U.C.R.N.T
PHENOMENA.ARENA
I.S.M...T.I.W.N
CUTTER.WINNIPEG
A.R.G...O.E.U.L
LAYMAN.INEDIBLE'
$echo"$grid"|lfreq-sh|column
1F3B4L5
M10O
15A
1K3C4U6
D11T
16N
1V3H4Y6
P13I
22E
1X3W5G6
S13R
TheScript
NL='
'
sed-e"s/./\\${NL}&/g"${1+"$@"}|
sort|
grep.|
uniq-c|
sort-n
Notes
Sincethisscriptwaswrittentoanalyzethelettersinacrossword
gridinwhichallthelettersarecapital,upper-andlowercaseletters
arenotcombined.
Ifyouwanttocountupper-andlowercaseasthesameletter,
addthislinebeforesort:
tr'[a-z]''[A-Z]'|
Anothermethodistoaddthe-foptiontosortandthe-ioption
touniq.
Ifyoulike,youcanalsoaddaccentedcharacterstothe
translationmix.Newerversionsoftrwilldoit(ifthelocaleisset
correctly)withthis:
tr'[:lower:]''[:upper:]'|
2.13fed—ASimpleBatchFileEditor
Onefrequentlyneedstomakeachangeoveroneormorefiles.It
canbetediousworkwhendonemanually,butascriptcandoitin
theblinkofaneye.
HowItWorks
Whileasimplescriptcannotduplicateallthecomplexitiesofawkor
sed,somecommontasksareeasilyautomated.Thefileeditor,fed,
candosimpletasks:Itcandeletelines,orperformasearchand
replace.Itcanoperateononeormorefiles.Testmodeshowsthe
differencesbetweenthefileandtheeditedversioninsteadof
replacingtheoriginalfile.
Usage
fed[OPTIONS]FILE[...]
-dRE:Deletelinescontainingregular
expression,RE,replacethemwithSTRif-r
isused.
-sRE:Searchpattern(regularexpression).
-rSTR:ReplacesearchpatternwithSTR.
-t:Testmode—showchangesthatwouldbe
made(usingdiff)insteadofreplacingthe
file.
Hereareafewexamples:
##Deletealllinesbeginningwithan
octothorpe(#)
##infileswhosenamesendwith.txt
fed-d'^#'*.txt
##Replacealllinesbeginningwith
anoctothorpe(#)with"##Commentdeleted"
##inallfileswhosenamesendwith
.sh
fed-d'^#'-r"##Commentdeleted"
*.sh
##Replaceallinstancesof4
consecutivelowercaseletterswithasingle"X"
fed-s'[a-z][a-z][a-z][a-z]'-rX
*.txt
##Deletealllinescontaininga
number,butjustshowthechanges
##thatwouldbemade,leavingthe
originalfilesintact
fed-td"[0-9]"$FILES
Remembertoquotespacesandwildcardcharactersinanyofthe
strings(andthepatternsifthecharactersaretobematched
literally).
TheScript
escape_slash(){
string=$1
"")string=;;
[2-9]|[1-9][0-9])string=${string//\//\\/};;
*)string='echo"$string"|sed's|/|\\/|g'`;;
esac
}
version="1.0"
progname=${0##*/}
delete_str=
replace_str=
search_str=
test=0
sed_cmd=
whilegetoptsvVh-:s:r:d:tvar
do
case$varin
d)search_str=.*$OPTARG.*;;
r)replace_str=$OPTARG;;
s)search_str=$OPTARG;;
t)test=1;;
v)verbose=$(($verbose+1));;
esac
done
shift$(($OPTIND-1))
case$verbosein
2)set-v;;
3)set-x;;
esac
[-n"$search_str"]&&{
escape_slash"$search_str"
search_str=$string
escape_slash"$replace_str"
sed_cmd="-e's/$search_str/$string/g'"
}
[-z"$sed_cmd"]&&{usage;exit5;
}
exit5
forfile
do
eval"sed$sed_cmd\"$file\">$tempfile"
if[$test-ge1]
then
printf"\n%s:\n""$file"
diff"$file""$tempfile"||printf"\n"
else
fi
done
[-f"$tempfile"]&&rm"$tempfile"
Summary
Thesescriptshavelookedatgenerictextfiles.Thecontentshave
beenirrelevant(lineendingsareformatting,ratherthancontentin
ouropinion).Someofthese,becausetheyarenotspecifictoany
onefileformat,willshowupinotherchaptersascommandsused
bythosescripts.
Manyotherscriptsthroughoutthebookdealwithfile
manipulation,fromstandardUnixconfigurationfilestoobscure
formatsforcrosswordpuzzles.Somewillbeusedtoextractand
presentinformationfromfiles,otherswillmanipulatethefiles
themselves,orcreatenewones.
Besidesbeingusefulontheirown,thesescriptscanbemodified
toapplytodifferentsituations.Feelfreetoexperiment,but,as
always,experimentoncopiesofyourfiles!
Footnotes
1 http://www.opengroup.org/onlinepubs/007908799/xbd/glossary.html#tag_004_000
(1) Ontario,Canada
WhenIwasintroducedtoBourneshellscripting,Itooktoit
immediately,despitesomeobviousdeficienciesinthelanguage.
Oneofthefeaturesmissingwasstringmanipulation.Itexisted,but
almosteverythinghadtobedonewithexternalcommandsthatwere
loadedfromthediskeachtime.TheKornShellintroducedanumber
ofparameterexpansionmodifiersthatdrasticallyreducedthe
numberofcallstoexternalcommands.Thesewerecopiedbyother
shellsandincorporatedintothePOSIXstandard.
TotakefulladvantageofthePOSIXfeatures,allthesescripts
arepresentedasfunctionsthatarecomprisedintwolibraries,charfuncsandstring-funcs.Sourcingthesefilesinyour.profile
makesthemavailableatthecommandline.(Replace.profilewith
whateverfileyourshelluses.)Manyofthesefunctions,however,
willrarelybeusedexceptinlargerscripts,andtheywillbeusedin
programsthatappearlaterinthebook.Stringsaregenerallyasubset
ofcharactersallowedintextfiles.Formostpracticalpurposes,they
containonlyprintablecharacters.Mostofthefunctionsinthis
chapterareintendedforuseonlywithASCIIcharactersintherange
1to126,thoughmostwillworkwithanextendedASCIIcharacter
set.
Traditionally,eachcharacterthatappearsonthescreenis
containedinan8-bitbyte.Sinceeachbitcanbeeither1or0,abyte
has256possiblevalues.TheASCIIcharactersetisasetof7-bit
numbers,containing128possiblevalues,thatrepresentcontrol
codes,letters,numbers,andpunctuation.Thecharacters0to31are
controlcodes,48to57arethedigits0to9,65to90arethecapital
letters,and97to122thelowercaseletters.Punctuationisscattered
throughtheremainingvaluesinthe0-to-127range.
Theremaining128values,from128to255,areusedinvarious
“extendedASCII”codings,suchastheDOSline-drawing
charactersortheISO8559-1LATIN-1characterset,whichcontains
theaccentedcharactersusedbywesternEuropeanlanguages.
CharacterActions:Thechar-funcsLibrary
Manyoperationsarerequiredonindividualcharacters—the
buildingblocksofstrings.Inthissectionyouarepresentedwith
functionsthatconverttoandfromnumericvalues,thatcapitalize
lettersandthatfindthenextcharacterinsequence.Thesefunctions
convertsinglecharacters.Allworkwithmultiplearguments,but
maynotbeefficientwhenusedonalargenumberofthem.Their
operationsarealldoneonsinglecharacters,notstrings.Ifan
argumenthasmorethanonecharacter,onlythefirstisused.
3.1chr—ConvertaDecimalNumbertoan
ASCIICharacter
Asthischapterwasbeingputtogether(firstedition),someoneinthe
comp.unix.shellnewsgroupaskedhowtoconvertanumbertoits
characterrepresentation.TheASCIIcharacter65isacapitalA;32is
aspace;49isa1.That’sallverywell,buthowdoesoneconvert65
toA?
HowItWorks
IpostedafunctionthatIhavebeenusingforalongtime.Ithought
thatitwouldbegoodtoincludeithere,alongwithanumberof
otherconversionfunctions.Unfortunately,Iranintoproblemswhen
Itesteditwithothershells,andonothersystems.(Chris)
Note
WhileChrisstartedoffwiththe8-bitCommodore,my8-bit
machinewastheSinclairZXSpectrum.OneofthethingsthatI
recollectaboutthesimplestwaytoturncapitalizationonoroff
wastoalterthe6thbit.The6thbitis25whichis32.UppercaseA
isASCII65andlowercaseaisASCII97.
Iwroteabouthalfadozenvariations,butonlyoneworkedonall
systemsandwithallshells.Surprisingly,consideringthatitusesan
externalprogram,awk,itisfasterthanmostshell-onlysolutions.
Usage
chr[-sS][-n]NN[...]
Theargumentsareintegersbetween1and255inclusive,andare
convertedtothecorrespondingASCIIcharacter.The-soptionis
followedbyaseparatorstringthatisprintedaftereachcharacter.By
default,thereisnoseparator,andthecharactersareruntogetherasa
word.
The-noptionsuppressesthefinalnewline,ausefulfeature
whencalledmultipletimesfromanotherscript.
$chr11599114105112116script
$chr$((119-32))114105116101
114
Writer
$chr-s.726576
H.A.L.
TheScript
chr()
{
##defaultscanbechangedbycommand-lineoptions
sep=##nothingisinsertedbetweencharacters
nl='\n'##anewlineisprintedafterallarguments
areprocessed
##Parsecommand-lineoptions
OPTIND=1
whilegetoptss:nvar
do
case$varin
s)sep=$OPTARG;;##separatorstringto
placebetweencharacters
n)nl=;;##suppressnewline
esac
done
shift$(($OPTIND-1))
##pipearguments,1toaline,intoawk,which
convertsthenumbers
##intotheequivalentcharacter
printf"%s\n""$@"|awk'{printf"%c%s",$1,sep}'
sep="$sep"
printf"%b""$nl"
}
3.2asc—ConvertaCharactertoItsDecimal
Equivalent
WhatnumberrepresentstheASCIIcharacterM?Whatis*?How
canIfindtheASCIIcodeforanygivencharacter?
HowItWorks
ThePOSIXspecificationoftheprintfcommandhastheanswer:
Iftheleadingcharacterisasingle-quoteordouble-quote,the
valueshallbethenumericvalueintheunderlyingcodesetof
thecharacterfollowingthesingle-quoteordouble-quote.
Forthesakeofconvenience,Ihaveencapsulatedthecodeinthe
ascfunction.
Usage
ascC[...]
Onlythefirstcharacterofeachargumentisconverted.
$ascasc
97
$ascaseriousquestion
97
115
113
Remembertoquotecharactersthathavespecialmeaningtothe
shell:
$asc"*"
42
TheScript
asc()
{
##loopthroughtheargumentsonthecommandline
forchar
do
##printtheASCIIvalueofthefirstcharacterof
$char
printf"%d\n""'$char"2>/dev/null
done
}
Notes
Possibleadditionstoascincludespecifyingaseparationcharacter,
asdemonstratedinthechrfunction;the$sepvariablewouldreplace
\n:
printf"%d${sep:-\\n}""'$char"
Thefunctioncouldthenbecalledwith
$sep=''ascACE
656769
3.3nxt—GettheNextCharacterinASCII
Sequence
Generatingthenextnumberinsequenceiseasy;justaddonetothe
previousnumber.Itisnotassimpletodothesamethingwith
letters,suchaswhennamingaseriesoffiles.Ifyouhavefilepic1a,
thenextfilewillbepic1b;ifyouhavepiano-P.ogg,thenitwould
bepiano-Q.ogg.Ifyouneedtochangeonlyoneortwofiles,you
wouldn’tmindabitoftyping;ifyouhavehundreds(oreven
dozens)offiles,youwillneedascripttodoit.
HowItWorks
Therearevariouswaystodoit,andyou’llfindothermethodslater
inthebook.Fornow,we’llusethenxtfunction.Thisscriptallows
younotonlytoderivethenextletter(orothercharacter)in
sequence,butalsotoreturnacharacteranynumberofplaces
forward—orbackward—intheASCIIcharacterset.
Usage
nxt[+|-N]C
Bydefault,nxtdeliversthenexthighercharacter,but,withan
optionalargument,itcanprintacharacteratanyoffsetfromtheone
given.Herearesomeexamples:
$nxtH
I
$nxt+5a
f
Anegativeoffsetprintsearliercharacters:
$nxt-9now
e
f
n
Sincetheoutputofnxtisproducedbythechrfunction,itis
handytohaveamethodtospecifyoptionstocontrolthewaythat
functionprintsitsresults.Theisdonewiththechr_optsvariable.If
weareconvertingseveralcharacters,itislikelythatwedonotwant
thedefaultbehavior:
$nxtHAL
I
B
M
Rather,allthelettersshouldappearonthesameline.Todothat,
wewantnxttocallchrwiththe-noption.Withthe-soption,we
canspecifyaseparatorcharacterforchrtoprintaftereach
conversion.Bydefiningchr-optsaheadofnxtonthesameline,its
valuewillonlyapplyduringthecalltonxt:
$chr_opts="-ns."nxtHAL
I.B.M.
TheScript
nxt()
{
##Thedefaultincrementis1,thenextcharacter
inc=1
case$1in
[+-]*[!0-9]*);;##noincrementis
specified,sousethedefault
[+-][0-9]*)inc=$1##incrementlinemaybe
positiveornegative
shift##removeincrement
;;
*);;
esac
forascin$(asc$*)##loopthroughthe
convertedarguments
do
chr$chr_opts$(($asc+${inc#+}))##convert
incrementednumbertocharacter
done
printf"\n"
}
3.4upr—ConvertCharacter(s)toUppercase
Eventhoughmostshellshavecommand-linekeycombinationsthat
convertawordtocapitalletters,orjustcapitalizethefirstletter,
thesedonotoperateonthecontentsofvariables;theychangethe
actualcharactersonthecommandline;thatis,thenameofthe
variable,notitscontents.Toconvertthecontentsofavariable,a
script(orotherutility)isneeded.
HowItWorks
IntheASCIIcharacterset,capitallettersare32characterstotheleft
ofthelowercaseletters.Thenxtcommand(describedpreviously)
withanoptionof-32willconvertalettertouppercase.Oruseupr,
whichisfaster,usesonlythefirstletterifawordisgivenasan
argument,andonlyconvertslowercaseletters.
UsingtheconventiondescribedinChapter1,thiscommand
comesasapairoffunctions.Onebeginswithanunderscore,and
storestheresultinavariable;thesecondcallsthefirstandprintsthe
contentsofthatvariable.Inuprasinanumberofothersuchpaired
functions,thesecondfunctioncantakemultiplearguments.
Usage
_uprc
uprc[...]
Here’sanexample:
$uprabc
A
B
C
Sinceupronlyconvertsthefirstcharacter,itiseasytocapitalize
words:
$b=bash
$_upr$b
$printf"%c%s\n""$_UPR""${b#?}"
Bash
TheScript
_upr()
{
##alook-uptableisthefastestmethod
case$1in
a*)_UPR=A;;b*)_UPR=B;;
c*)_UPR=C;;d*)_UPR=D;;
e*)_UPR=E;;f*)_UPR=F;;
g*)_UPR=G;;h*)_UPR=H;;
i*)_UPR=I;;j*)_UPR=J;;
k*)_UPR=K;;l*)_UPR=L;;
m*)_UPR=M;;n*)_UPR=N;;
o*)_UPR=O;;p*)_UPR=P;;
q*)_UPR=Q;;r*)_UPR=R;;
s*)_UPR=S;;t*)_UPR=T;;
u*)_UPR=U;;v*)_UPR=V;;
w*)_UPR=W;;x*)_UPR=X;;
y*)_UPR=Y;;z*)_UPR=Z;;
*)_UPR=${1%${1#?}};;
esac
}
upr()
{
_UPR=
##convertcharacter[s]touppercase
forchr
do
_upr"$chr"
printf"%s${sep:-\\n}""$_UPR"
done
printf"${_UPR:+\n}"
}
Notes
Asscriptsgo,_uprisinelegant,butitisfast,andonceitishidden
awayinalibrary,youneverneedseeitagain.Ashorterbutslower
solutionistouseawrapperforthenxtfunction:
upr()
{
##loopthroughthecommand-linearguments
forchr
do
case$chrin
[a-z]*)chr_opts="-n"nxt-32"$chr";;
*)printf"%c${sep}""$chr"2>/dev/null;;
esac
done
}
3.5lwr—ConvertCharacter(s)toLowercase
Thetrcommandcanconvertcharacterstolowercase,butan
externalcommandisaninefficientwaytoconvertasingle
character.Ashellfunctionismuchfaster.
HowItWorks
Asimplelook-uptableisthefastestwaytoconvertasingle
characterfromuppercasetolowercase.
Usage
_lwrC##setsthe_LWRvariable
lwrC[...]##printstheresult(s)
Ifyouwanttoassigntheresulttoavariable,usetheunderscore
version:
$_lwrX
$var=$_LWR
Thesecondversionisforprintingtheresult:
$lwrW
w
TheScript
_lwr()
{
_LWR=
case$1in
A*)_LWR=a;;B*)_LWR=b;;
C*)_LWR=c;;D*)_LWR=d;;
E*)_LWR=e;;F*)_LWR=f;;
G*)_LWR=g;;H*)_LWR=h;;
I*)_LWR=i;;J*)_LWR=j;;
K*)_LWR=k;;L*)_LWR=l;;
M*)_LWR=m;;N*)_LWR=n;;
O*)_LWR=o;;P*)_LWR=p;;
Q*)_LWR=q;;R*)_LWR=r;;
S*)_LWR=s;;T*)_LWR=t;;
U*)_LWR=u;;V*)_LWR=v;;
W*)_LWR=w;;X*)_LWR=x;;
Y*)_LWR=y;;Z*)_LWR=z;;
*)_LWR=${1%${1#?}};;
esac
}
lwr()
{
##convertcharacter[s]tolowercase
forchr
do
_lwr"$chr"&&printf"%s""$_LWR"
done
printf"${_LWR:+\n}"
}
Notes
Thismorecompactversion,usingthenxtfunctionfromearlierin
thechapter,maylookbetter,butisfarslower.
lwr()
{
##convertcharacter[s]tolowercase
forchr
do
case$chrin
[A-Z]*)chr_opts="-n"nxt+32"$chr";;
*)printf"%c""$chr";;
esac
done
}
StringCleaning:Thestring-funcsLibrary
Extracting,removing,andreplacingsectionsofastringare
everydaytasks.Thefunctionsintheprevioussectionoperatedon
characters;inthissection,theyactonwholestrings.(Mostofthese
functionscomeinpairs,onewhichperformstheoperationand
storesitinavariable,theothercallingthefirstfunctionandprinting
theresult;thisisdescribedinmoredetailinChapter1.)
3.6sub—ReplaceFirstOccurrenceofaPattern
Replacingonewordorcharacterwithanotherisamostbasic
operation.Whenchanginganentirefile,theUnixcommandsedis
theobviousmethodtouse.Ifyouaremakingthechangeinasingle
line,orevenasingleword,sed,becauseitisanexternalcommand,
isslowerthanashellfunction.
HowItWorks
Ifyouusethesubfunction,theparameterexpansioncapabilityof
thePOSIXshellisutilized,andnoexternalcommandisnecessary.
Notonlythat,butitcutsdownontheamountoftypingyouhaveto
do.Asanaddedbonus,youcanuseawildcardpatterntoselectthe
texttobereplaced.
Usage
subSTRINGPATTERN[TEXT]
IfastringmatchingPATTERNisfoundwithinSTRING,itsfirst
occurrencewillbereplacedbyTEXT.Toseehowitworks,first
assignavaluetothevariable,string:
string="Thequickbrownfox"
Here’sasampleshowingthecommandsastyped,alongwiththe
result:
$sub"$string"quicknervous
Thenervousbrownfox
$sub"$string""""_"
The_quickbrownfox
Bynotsupplyingareplacementtext,thePATTERNwillbedeleted
insteadofreplaced;moreaccurately,itwillbereplacedbyanull
string.
$sub"$string""quick"
Thebrownfox
TheScript
_sub()
{
_SUB=$1
[-n"$2"]||return1##nothingtosearchfor:
error
s_srch=${2}##patterntoreplace
rep=$3##replacementstring
case$_SUBin*$s_srch*)##ifpatternexistsin
thestring
sr1=${_SUB%%$s_srch*}##takethestring
precedingthefirstmatch
sr2=${_SUB#*$s_srch}##andthestring
followingthefirstmatch
_SUB=$sr1$rep$sr2##andsandwichthe
replacementstringbetweenthem
;;
*)return1;;##ifthepatterndoesnot
exist,returnanerrorcode
esac
}
sub()
{
_sub"$@"&&printf"%s\n""$_SUB"
}
Notes
Bothbash2+andksh93haveanadditionalformofparameter
expansionthatcanmakethisfunctionmuchshorter:
_sub(){
_SUB=${1/$2/$3}
}
Acorollaryfunction,rsub,replacesthelastinstanceofthe
patterninsteadofthefirst:
_rsub()
{
str=$1
["$2"]||return1
s_srch=${2}
rep=$3
case$strin*$s_srch*)
sr1=${str%$s_srch*}
sr2=${str##*$s_srch}
_RSUB=$sr1$rep$sr2
;;
*)_RSUB=
false;;
esac
}
rsub()
{
_rsub"$@"&&printf"%s\n""$_RSUB"
}
3.7gsub—GloballyReplaceaPatterninaString
Globallyreplacingatextpatterninastringwithanotherisasimple
jobforaPOSIXshell.Noexternalcommandisrequired.
HowItWorks
Thegsubfunctionusesthesamesyntaxasthepreviousscript,sub,
butitdoesn’tstopafteronereplacement.
Usage
gsubSTRINGPATTERN[TEXT]
IfastringmatchingPATTERNisfoundwithinSTRING,all
occurrenceswillbereplacedbyTEXT.We’lltrapafoxinthe
variable,string,todemonstrate:
string="Thequickbrownfox"
Oftenusefulforfixingfilenames,thiscommandreplacesallthe
spacesinstringwithunderscores:
$gsub"$string""""_"
The_quick_brown_fox
Notsupplyingareplacementtextcausesalloccurrencesof
PATTERNtobedeletedinsteadofreplaced.
$gsub"$string"""
Thequickbrownfox
Usingawildcardpatternwillreplaceallsubstringsthatmatch.
Forexample,tochangeanyletterfromatomwithQ:
$gsub"$string""[a-m]"Q
TQQquQQQQrownQox
Thewildcardcanbenegated,sothatallcharactersexceptthose
inthepatternwillbechangedordeleted:
$gsub"$string""[!a-m]"
heickbf
Howmanyosarethereinthestring?Bydeletingeverything
thatisnotano,wecancountwhat’sleft:
$_gsub“$string”“[!o]”
$echo${#_GSUB}
2
TheScript
_gsub()
{
##assignthestringtosr2;thiswillbegobbledup
witheachsubstitution
sr2=$1
##returnanerrorifthereisnopatternspecified
[-n"$2"]||return1
##assignthesearchpatterntos_srch
s_srch=${2}
##assignthereplacementtext,ifany,torep
rep=$3
##sr1willholdtheportionofthestringthathas
beenprocessed
sr1=
##loopuntilsr2nolongercontainsthesearch
pattern
while:
do
case$sr2in
*$s_srch*)
##addthestringuptothematch,
##andthereplacementtext,tosr1
sr1=$sr1${sr2%%$s_srch*}$rep
##removeuptothematchfromsr2
sr2=${sr2#*$s_srch}
;;
*)break;;
esac
done
_GSUB=$sr1$sr2
}
gsub()
{
_gsub"$@"&&printf"%s\n""$_GSUB"
}
Notes
Theaddedcapabilitiesofbash2+andksh93canreduce_gsubtoa
singleline:
_gsub(){_GUSB="${1//$2/$3}";}
3.8repeat—BuildaStringofaSpecifiedLength
Occasionallyyouneedacharacter,orastring,repeatedacertain
numberoftimes.Perhapsit’sonlyforprintingalineofdashes
acrossthewidthofyourscreen,orfordrawingpatterns,butthe
needarisesmoreoftenthanonemightexpect(forexample,seethe
substrfunctionlaterinthischapter).
HowItWorks
Withtherepeatfunction,stringscanrapidlybebuiltuptoany
desiredlength.Becausethestringtriplesinlengtheachtime
throughthewhileloop,evenlongstringsareproducedveryquickly.
Usage
_repeat[-n]STRINGLENGTH##result
isstoredin$_REPEAT
repeat[-n]STRINGLENGTH##result
isprinted
ThisfunctionproducesastringofLENGTHcharacters,or,ifthe-n
optionisused,withLENGTHrepetitionsofSTRING.Hereareacouple
ofexamples,withtheresults:
$repeat=30
==============================
$_repeat-n'/\'20
$printf"%s\n""$_REPEAT"
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
Here,repeatisusedinabricklayingdemonstration:
$_repeat-n'_____|'20
$printf"%80.80s\n""$_REPEAT"
"${_REPEAT#???}""$_REPEAT""${_REPEAT#???}"
_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|__
__|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____
_____|_____|_____|_____|_____|_____|_____|_____|_____|_____|__
__|_____|_____|_____|_____|_____|_____|_____|_____|_____|_____
TheScript
_repeat()
{
##Ifthefirstargumentis-n,repeatthestringN
times
##otherwiserepeatittoalengthofNcharacters
case$1in
-n)shift
r_num=$((${#1}*$2))
;;
*)r_num=$2
;;
esac
_REPEAT=$1
while[${#_REPEAT}-lt${r_num}]
do
_REPEAT=$_REPEAT$_REPEAT$_REPEAT
done
while[${#_REPEAT}-gt$r_num]
do
_REPEAT=${_REPEAT%?}
done
}
repeat()
{
_repeat"$@"&&printf"%s\n""${_REPEAT}"
}
Notes
Ihavetriedconcatenatingmoreinstancesof$_REPEATatonetime,
andfoundthat,usingbashonmysystem,threeistheoptimum
numberineachiterationofthewhileloop.Thismayvaryfromshell
toshellandsystemtosystem,asitdependsonthestringhandling
functionsusedbytheshell.Tryitwithmore,ifyoulike;your
systemmaybehavedifferently.
3.9index,rindex—FindPositionofOneString
WithinAnother
Whenaskedtoprovideascripttoconvertanabbreviatedmonth
name(Jan,Feb,etc.)tothenumberofthemonth,onemethodwto
useasinglestringcontainingallthemonths,andparameter
expansiontodeterminethelocationofthemonthwithinthestring:
mm=JanFebMarAprMayJunJulAugSepOctNovDec
idx=${mm%%$monthname*}
month=$(((${#idx}+3)/3))
Withthequestion"DoesstringAcontainStringB?,andifit
does,atwhatindexinthestring?,thesequestionslettotheresulting
functionandisbeenusedinanumberofscripts(later)inthebook.
HowItWorks
Dependingonwhetheryouwantthepositionofthefirstoccurrence
orthelast,orevenjustwhetheritisthereornot,oneoftheindexor
rindexfunctionswillprovidetheanswer.Thesefunctionsdetermine
thepositionofthesubstringinthemainstring;indexfindsthe
positionofthefirstoccurrence,andrindex,thepositionofthelast.
Usage
_indexSTRINGASTRINGB
indexSTRINGASTRINGB
_rindexSTRINGASTRINGB
rindexSTRINGASTRINGB
IfSTRINGBisnotcontainedinSTRINGA,bothfunctionsprint0;
otherwise,theyprintthepositionofthefirstorlastoccurrenceof
STRINGB.
$str="tobeornottobe"
$index"$str"be
4
$rindex"$str"to
14
Ifyouonlywanttoknowwhetheramatchwasfound,youcan
discardtheoutputofthesefunctions,andcheckthereturncode:
$if_indexqwertyz
>then
>echoStringfound
>else
>echoStringnotfound
>fi
Stringnotfound
$_indexqwertyr
$[$?-eq0]&&echoStringfound
||echoStringnotfound
Stringfound
wecouldhaveused_rindexintheseexamples,butinalazy
way,_indexisshorter.
TheScript
Ifyouusethe%%parameterexpansionoperator,thestringtotheleft
ofthesubstringisextracted,anditslengthplusoneistheposition
ofthesubstring.
_index()
{
case$1in
*$2*)##extractuptobeginningofthe
matchingportion
idx=${1%%$2*}
##thestartingpositionisonemore
thanthelength
_INDEX=$((${#idx}+1));;
*)_INDEX=0;return1;;
esac
}
index()
{
_index"$@"&&printf"%d\n""$_INDEX"
}
Ifyouuse%insteadof%%,thelastoccurrenceofthesubstringis
found,andthelengthofthestringuptothatpoint,plusone,isthe
positionofthelastoccurrenceofthesubstring.
_rindex()
{
case$1in
*$2*)##extractuptobeginningofthelast
matchingportion
idx=${1%$2*}
##thestartingpositionisonemore
thanthelength
_RINDEX=$((${#idx}+1));;
*)_RINDEX=0;return1;;
esac
}
rindex()
{
_rindex"$@"&&printf"%d\n""$_RINDEX"
}
3.10substr—ExtractaPortionofaString
ThePOSIXparameterexpansionoperators,${var%xx}and
${var#xx},workonthebeginningandendofstrings.However,
scriptsoftenneedtotakechunksoutofthemiddleofastring.For
example,aprogramspitsoutthedateandtimeintheformat
20050312204309.Youneedthemonthandtheday.Howdoyou
extractthemusingonlyshellcommands?
Consideranotherscenario:Anidentificationnumbercontainsa
classificationinthesecondandthirdcharactersfromtheendofa
numberthatmayhaveanywherefrom7to20lettersanddigits.
Whatdoyouusetopullthatclassificationoutofthelargernumber?
HowItWorks
Thesubstrfunctioncanextractanynumberofcharactersfrom
anywhereinthestring.(Itusesapreviousfunction,repeat,tobuild
thewildcardstringofquestionsmarksusedintheparameter
expansion.)
Usage
substrSTRINGFIRST[LENGTH]
ThestringthatsubstrreturnsbeginsattheFIRSTcharacterof
STRING.IfFIRSTisnegative,thepositioniscountedfromtheendof
thestring.IfthevalueofFIRSTisgreaterthanthelengthofthe
string,anerrorisreturned.
HerearetheanswerstotheproblemsI’veposed.Toextractthe
monthanddayfromthestringcontainingthedateandtime:
$_substr2005031220430952
$month=$_SUBSTR
$_substr2005031220430972
$day=$_SUBSTR
$printf“Month:%d\nDay:%d\n”
${month#0}${day#0}
Month:3
Day:12
Toextractthecharactersfromtheendofastring,anegative
numberisusedforthestartingposition:
$id=AZ3167434G34
$substr“$id”-32
G3
TheScript
Firstthebeginningofthestringhastoberemovedsothatthe
desiredportionstartsthenewstring.Thenparameterexpansionis
usedtoremovetheunwantedportionfromtheendofthestring.
_substr()
{
_SUBSTR=
##storetheparameters
ss_str=$1
ss_first=$2
ss_length=${3:-${#ss_str}}
##returnanerrorifthefirstcharacterwantedis
beyondendofstring
if[$ss_first-gt${#ss_str}]
then
return1
fi
if[$ss_first-gt1]
then
##buildastringofquestionmarkstouseasa
wildcardpattern
_repeat"?"$(($ss_first-1))
##removethebeginningofstring
ss_str=${ss_str#$_REPEAT}
elif[${ss_first}-lt0]##${#ss_str}]
then
##countfromendofstring
_repeat"?"${ss_first#-}
##removethebeginning
ss_wild=$_REPEAT
ss_str=${ss_str#${ss_str%$ss_wild}}
fi
##ss_strnowbeginsatthepointwewanttostart
extracting
##soprintthedesirednumberofcharacters
if[${#ss_str}-gt$ss_length]
then
_repeat"${ss_wild:-??}"$ss_length
ss_wild=$_REPEAT
_SUBSTR=${ss_str%${ss_str#$ss_wild}}
else
_SUBSTR=${ss_str}
fi
}
substr()
{
_substr"$@"&&printf"%s\n""$_SUBSTR"
}
Notes
Ashasbeendemonstratedinotherscripts,ashorter,fasterversion
isavailableforbash2+andksh93.
substr()
{
if[$2-lt0]
then
printf"%s\n""${1:$2:${3:-${#1}}}"
else
printf"%s\n""${1:$(($2-1)):${3:-${#1}}}"
fi
}
3.11insert_str—PlaceOneStringInside
Another
Inmanyplaces(especiallyfilenames),datesappearasa10-digit
numberrepresentingtheyearwithfourdigits,andthemonthand
daywithtwodigitseach.Sayyourpreferredformatforstoringa
dateisYYYY-MM-DD,youwillneedamethodofinserting
hyphensafterthefourthandsixthdigits.
HowItWorks
Usingthe_substrfunction,insert_strsplitsthestringintotwo
piecesandsandwichestheinsertbetweenthem.
Usage
_insert_strSTRINGINSERT[POSITION]
insert_strSTRINGINSERT[POSITION]
POSITIONisthecharacterpositionintheSTRINGatwhichtoplace
INSERT,countingfrom1.ToconvertadateinYYYYMMDDformat
tointernationalformat(YYYY-MM-DD),hyphensmustbeadded
atpositions7and5:
$date=20050315
$_insert_str"$date"–7
$insert_str"$_INSERT_STR"–5
2005-03-15
TheScript
_insert_str()
{
_string=$1##The(soon-to-be)containerstring
i_string=$2##Thestringtobeinserted
_pos=${3:-2}##defaulttoinsertingafterfirst
character(position2)
##Storethestringuptothepositionoftheinsert
_substr"$_string"1$(($_pos-1))
i_1=$_SUBSTR
##Storethestringthatwillgoaftertheinsert
_substr"$_string"$_pos
i_2=$_SUBSTR
##Sandwichtheinsertbetweenthetwopieces
_INSERT_STR=$i_1$i_string$i_2
}
insert_str()
{
_insert_str"$@"&&printf"%s\n""$_INSERT_STR"
}
Notes
Forbash2+andksh93,thereisafasterversion:
_insert_str()
{
_string=$1
i_string=$2
i_c=${3:-2}##usedefaultifpositionnotsupplied
i_1=${_string:0:i_c-1}
i_2=${_string:$i_c}
_INSERT_STR=$i_1$i_string$i_2
}
Summary
Thesecharacterandstringfunctionsonlyscratchthesurfaceofthe
manipulationspossiblewiththeshell,andmanymoreappearlater
inthebook.Inthischapteryouarepresentedwithbasicfunctions
thatprovidebuildingblocksformorecomplexscripts.Wehave
alreadyseenfunctionsusedinotherfunctions(_repeatin_substr,
and_substrin-insert_str,forexample);agreatdealmorewillbe
providedasthebookprogresses.
(1) Ontario,Canada
ThemostpopularpageonChris'website(
http://cfaj.freeshell.org/)istheWordFinder.Ithasprograms
forfindingwordsthatfitapatterninacrosswordgrid(orona
Scrabbleboard).Itcanfindanagrams,includinganagramsofthe
wordifanotherletterisaddedtoit,orifoneisremovedfromit.It
canfindwordsthatbeginwith,endwith,orcontainanysequenceof
letters(seeFigure4-1).
Figure4-1. TheWordFinderwebpage
ThesebeganlifeasCprogramsonhisCommodoreAmiga,
almost20yearsago.Combinedwithmacrosinatexteditor,they
constitutedhissystemforcomposingcrosswordpuzzles.Then,ata
timewhenhecouldn’taffordtoreplaceit,hisharddrivecrashed.
Thoughhehadeverythingbackeduptofloppydisks,andtheAmiga
couldberunfromfloppies,buthiswordlistswouldn’tfitona
floppy.Eveniftheyhad,itwouldhavebeenexcruciatinglyslowto
use.
Afewweeksbeforethecrash,hehadbuilthisfirstwebsite(the
officialsiteofthe1996CanadianClosedChessChampionships).It
wasrunningonaUnixmachine,andhehadshellaccess.TheC
sourcecodefromtheAmigawasuploadedtothewebsite,andmost
ofitcompiledwithoutahitch.Whatdidn’t,wasrewrittenasshell
scripts.Whenthiswasmovedtoitscurrentlocationacoupleof
yearslater,hewroteeverythingasashellscriptandaddedsome
newfeatures.
Sometimethereafter,theAmigawasretiredandGNU/Linux
wasthenewdesktop.Thecentralcrosswordprogramheusesisstill
writteninC,butanynewutilitiesareshellscripts,oftenaccessed
throughemacsmacros.Thischapterprovidesacollectionofscripts
forplayingwithwords.
FindingandMassagingWordLists
Mostofthescriptsinthischapteruseawordlist.Unixsystems
usuallycomewithawordlistin/usr/share/dictor
/usr/local/dict,buttheselistsaredesignedforsystem
administration(forexample,checkingthatpasswordsarenotbased
ondictionarywords)andspellchecking;theyvarygreatlyinsize
andfitnessforourpurposes.ForseveralyearsChrishasbeenusing
TheUKAdvancedCrypticsDictionary(UKACD),compiledby
RossBeresford.Thereareseveralothers,somelarger,some
smaller.
Thewidelyavailablefiles,web2.txtandweb2a.txt,arederived
fromWebster’sNewInternationalDictionary,SecondEdition.
Theseandmorelists,includingtheOfficialScrabblePlayers’
Dictionary,areavailablefordownloadontheNationalPuzzlers’
Leaguewebsite,
http://www.puzzlers.org/secure/wordlists/dictinfo.php.
Beforeanyoftheselistscanbeused,somepreparationis
needed.ThesescriptsdotheconversionsnecessaryfortheUKACD,
andwillworkwithanywordlist.
wf-funcs:WordFinderFunctionLibrary
ThislibrarydefinesafewfunctionsneededbytheWordFinder
utilities,aswellassourcingotherfunctionlibrarieswith
.string-funcs
Thislibrary,describedinChapter3,definesanumberof
functionsandloadstwootherlibraries,standard-funcs(Chapter1),
andchar-funcs(Chapter3).
4.1write_config—WriteUser’sInformationto
theConfigurationFile
WordFinderneedstoknowwheretolookforitswordfiles.These
needtobeplacedinaconfigurationfile.
HowItWorks
ThedirectoriesthatWordFinderneedsarestoredin$configfile;
write_configstorestheuser’sinformationinthatfile,whichmay
bethedefault,$HOME/.config/wordfinder.cfg,oronespecifiedon
thecommandlineofthecallingscript.
Usage
write_config
Therearenooptionsorargumentstothisfunction,whichis
normallycalledbywf-setup.
TheScript
write_config()
{
{
printf"\n%s\n""##Thisfile"
printf"configfile=\"%s\"\n""$configfile"
printf"\n%s\n""##Directorycontainingmain
wordlists"
printf"dict=\"%s\"\n""$dict"
printf"\n%s\n""##Directorywhereuserplaces
wordfiles"
printf"%s\n""##(maybethesameas\$dict)"
printf"userdict=\"%s\"\n\n""$userdict"
}>"$configfile"
}
4.2do_config—CheckForandSourceDefault
ConfigurationFile
Ifnoconfigurationfileissuppliedonthecommandline,the
WordFinderutilitiesneedtouseadefaultfile.Ifthefiledoesnot
exist,itneedstobegenerated.
HowItWorks
Ifthedefaultconfigurationfile,$HOME/.config/wordfinder.cfg,
doesnotexist,theuserisaskedwhetheritshouldbecreated.Ifthe
userchoosesYes,thewf-setupscriptisrun.Thefileisthensourced,
ifitexists.
Usage
do_config||exit5
Ifthefunctionfails,thenormalactionistoexitthescriptwith
anerror.
TheScript
do_config()
{
[-d$HOME/.config]||mkdir$HOME/.config||
return9
wf_cfg=${wf_cfg:-$HOME/.config/wordfinder.cfg}
[-f"$wf_cfg"]&&."$wf_cfg"||{
##ifstdinisnotconnectedtoaterminal,
##checkforthefilesandreturnsuccessor
failuure
[-t0]||{
[-d/usr/share/dict]&&{
dict=/usr/share/dict;return;}
[-d/usr/local/dict]&&{
dict=/usr/local/dict;return;}
return2
}
printf"\n%s:couldnotfindconfigfile,
%s\n""$progname""$wf_cfg"
printf"Doyouwishtorunthesetup
programnow[Y/n]?"
get_keyQ
case$Qin
""|y|Y|"$NL")wf-setup;;
*)printf"\n";return5;;
esac
."$wf_cfg"
}
}
4.3set_sysdict—SelecttheDictionaryDirectory
Thedirectorythatcontainsthewordlistswillusuallybe
/usr/share/dictor/usr/local/dict,butiftheuserdoesn’thave
permissiontowritetooneofthosedirectories,anotherdirectory
mustbeused.
HowItWorks
Ifeitheroftheusualdictionarylocationsisfound,theuserisasked
whethertousethatdirectoryorchooseanother.
Usage
set_sysdict||exit5
Ifthefunctionfails,thecallingscriptshouldusuallyexitwithan
error.
TheScript
set_sysdict()
{
##Thedictdirectoryisnormallyinoneoftwo
places.
##Ifeitherisfound,itbecomesthedefault
directory
forsysdictin/usr/share/dict/usr/local/dict""
do
[-d"${sysdict}"]&&break
done
##Theuserisaskedtoconfirmthedirectory
alreadyfound,
##orchooseadifferentone.
while:
do
printf"Enternameofdirectoryforword
lists${sysdict:+($sysdict)}:"
readd
if[-n"$d"]&&[-d"$d"]
then
sysdict=$d
break
elif[-z"$d"]
then
break
else
printf"%sdoesnotexist\n""$d"
fi
done
echo"Mainwordlistdirectory:$sysdict"
}
4.4mkwsig—SortLettersinaWord
Howcanyoutellifonewordisananagramofanother?Howcan
youfindanagramsofaword?
HowItWorks
IfwordAisananagramofwordB,thensortingthelettersofeach
wordalphabeticallywillproduceidenticalresults.Thelettersof
bothTRIANGLEandINTEGRAL,whensorted,yieldAEGILNRT.
ThesameistrueofRELATING,ALTERING,ALERTING,and(yes,
itisaword)TANGLIER.Theyareallanagramsofeachother.
Thefunction,mkwsig,convertsthelettersofawordtouppercase
andsortsthem.Thisworkswellfordealingwithonewordatatime,
butitisreallyslowwhenprocessing200,000words,aswhen
buildingthefilenecessarytofindanagrams.Thewf-setupscriptat
theendofthissectionusesit(andpointstoafasteralternative),but
goouttodinnerwhilewaitingforittofinish.
Usage
mkwsigWORD
Thisfunctionisusedprimarilybytheanagramscript,butitcan
beusedonitsown(and,ofcourse,inotherscripts):
$mkwsigdared
ADDER
TheScript
##TheassignmenttoNLisnot
necessaryifstandard-varshasbeensourced
NL='
'
mkwsig()
{
echo"$1"|##Thewordissuppliedon
thecommandline
tr'[a-z]''[A-Z]'|##Convertalllettersto
uppercase
sed"s/./&\\$NL/g"##Puteachletterona
separateline
sort|##Sorttheletters
alphabetically
tr-d'\012'##Removethenewlines
}
4.5wf-clean—RemoveCarriageReturnsand
Accents
TheUKACDfilehasCR/LFlineendingsandaccentedcharacters.It
alsocontainscompoundwords.Weneedtocleanupthefileand
splititintotwofiles.
HowItWorks
ThisscriptusestrtoconvertaccentedcharacterstoASCII,thensed
removesthecarriagereturnsanddeleteswordsthatbeginorend
withcharactersotherthanletters.Theseareprefixes,suffixes,and
contractions(suchas’tis).Theresultingfileissplitintotwofiles,
singlewordsandcompounds.
Usage
wf-clean[FILE]
IfnoFILEisgivenonthecommandline,theuserispromptedto
enterafilename.
TheScript
description="Convertaccented
characterstoASCIIandremovecarriagereturns"
progname=${0##*/}
.standard-vars##loadvariable
definitions,includingCR
die()
{
exitcode=$1
shift
if["$*"]
then
echo"$*"
fi
exit${exitcode:-1}
}
file=${1:-`printf"Enterfilename:"
>&2;readfile;echo$file`}
[-f"$file"]||die5"$progname:
File($file)doesnotexist"
w
##setlistofaccentedcharacters
andtheirequivalentASCIIcharacter
accents="ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏ_ÑÒÓÔÕÖØÙÚÛÜ_àáâãäåçèéêëìíîïñòóôõöøùúû
ascii="AAAAAACEEEEIIIIDNOOOOOOUUUUYaaaaaaceeeeiiiinoooooouuuuy
##Convertaccentedcharactersto
ASCII,
##removecarriagereturns,
##anddeletewordsthatdonotbegin
orendwithanASCIIcharacter
tr"$accents""$ascii"<"$file"|
sed-e's/$CR//'\
-e'/^[â-zA-Z]/d'\
-e'/[â-zA-Z]$/d'>"$file.clean"
##storeanywordsthatcontainnonASCIIcharactersinthecompoundsfile...
grep'[â-zA-Z]'"$file.clean">
compounds
##...andtherestinthe
singlewordsfile
grep-v'[â-zA-Z]'"$file.clean">
singlewords
4.6wf-compounds—SquishCompoundWords
andSavewithLengths
Whenenteredincrosswordpuzzles,compoundwordshaveno
spacesorhyphens.Therefore,theirlengthsinthewordlistarenot
thesameastheirlengthsinthepuzzlegrid.Weneedtofixthat,and
provideameansofretrievingtheoriginalwordorphrasealongwith
thelengthsastheyappearinacrypticcrosswordpuzzle.
HowItWorks
Toconvertphrasessothattheyarethesamelengthaswhenina
puzzle,aswellastogivethelengthsastheywouldappearina
crypticcrosswordpuzzle,wesquishthewordstogether.Tomake
thesquishedwordsmorereadable,thefirstletterafteraspaceor
hyphenhasbeenremovediscapitalized.Apostrophesareremoved,
buttheydonotaffectthelettercount.
Thewordsareprintedinthreetab-separatedfields:thesquished
words,theoriginalphrase,andthewordlengths.Hereareafew
examples:
AladdinsLampAladdin’slamp(8,4)
blueGreenAlgaeblue-green
algae(4-5,5)
swordOfDamoclesswordof
Damocles(5,2,8)
wireHairedTerrierwire-haired
terrier(4-6,7)
Usage
wf-compounds[FILE][>newfile]
IfFILEisnotgiven,afilenamedcompoundswillbeusedifit
existsinthecurrentdirectory.
TheScript
Thesquishingandgenerationoftheword-lengthsstringisdoneby
thesquishfunction,whichiskeptinthewf-funcslibrary:
squish()
{
phr=$1
lengths="("
lastleftlen=0
while:
do
case$phrin
##removeapostrophes
*\'*)phr=${phr%%\'*}${phr#*\'};;
##ifanynon-alphabeticcharactersstill
exist
*[!a-zA-Z]*)
##removenon-alphabeticcharacters
##andconvertfollowingletterto
uppercase
left=${phr%%[!a-zA-Z]*}
right=${phr#$left}
char=${right%${right#?}}
right=${right#?}
leftlen=$((${#left}-$lastleftlen
))
lastleftlen=${#left}
while:##removeconsecutivenonalphabeticcharacters
do
case${right%${right#?}}in
[!a-zA-Z])right=${right#?}
;;
*)break;;
esac
done
##Buildthelengthsstring,using
eitheracommaorahyphen
case$charin
'-')lengths=$lengths$leftlen-;;
*)lengths=$lengths$leftlen,;;
esac
_upr"$right"
phr=$left$_UPR${right#?}
;;
##Allcleanedup;exitfromtheloop
*)break;;
esac
done
lengths="${lengths}${#right})"
}
Thewf-compoundscommandsourceswf-funcs,whichinturn
sourcesstandard-funcs(fromChapter1)andstring-funcs(from
Chapter2).
.wf-funcs
##Ifthewfconfigcommanddoesnot
exist,createadummyfunction
typewfconfig>/dev/null2>&1||
wfconfig(){noconfig=1;}
verbose=0
progname=${0##*/}
file=compounds
##parsecommand-lineoptions
whilegetoptsvVf:var
do
case$varin
f)file=$OPTARG;;
esac
done
shift$(($OPTIND-1))
[-f"$file"]||die5"$progname:
$filenotfound"
##iftheverboseoptionisused,
printadotforevery1000wordsprocessed
case$verbosein
[1-9]*)dotty=`repeat".."$((1000/$verbose))`
dot()
{
case$dotin
"$dotty")
dot=
printf".">&2
;;
*)dot=$dot.;;
esac
}
;;
*)dot(){:;};;
esac
##Readeachlineofthefile,call
squish(),andprinttheresults;
##theoutputshouldberedirectedto
thecompoundsfile
whileIFS=read-rline
do
squish"$line"
printf"%s\t%s\t%s\n""$phr""$line""$lengths"
dot
done<$file
4.7wf-setup—PrepareWordandAnagramLists
Theconversionsinthepreviousscriptsallmustbeperformed,and
theresultingfilesinstalledinthecorrectdirectory.
Thewf-setupscriptaskstheusertoverifytheinstallation
directory,thenpullstogetherthepreviousscriptsandinstallsthem.
Usage
wf-setup[-cconfigfile]
Unlessyouwanttouseanonstandard.cfgfile,justrunwfsetupandansweranyquestionsitasks.
TheScript
description="Preparewordlistsfor
searchesandanagrams"
progname=${0##*/}
##ifthecommand’snameendsin-sh,
it’sthedevelopmentversion
##sousethedevelopmentversionsof
thefunctionfiles
case$prognamein
*sh).wf-funcs-sh;;
*).wf-funcs;;
esac
configfile=$HOME/.config/wordfinder.cfg
wordfiles="
singlewords
Compounds
singlewords.anag
Compounds.anag
"
whilegetoptsVc:var
do
case$varin
c)configfile=$OPTARG;;
esac
done
shift$(($OPTIND-1))
printf"\n\n"
set_sysdict||exit5
write_config
ifcheckfiles${sysdict}$wordfiles
then
printf"\n\tAllrequiredfilesfoundin${sysdict}\n"
exit
fi
[-f$sysdict/singlewords]||wf-clean
[-f$sysdict/Compounds]||
wf-compounds"$sysdict/compounds">
$sysdict/Compounds
##Iswordsortinstalled?(Itis
availablefrom
http://cfaj.freeshell.org
)
typewordsort&&
["`wordsort--version`"="--EINORSV$TAB--version"]
&&
ws=1||ws=0
forfileinsinglewordsCompounds
do
[-f"$sysdict/$file.anag"]||{
if["$ws"-eq1]
then
cut-f1"$sysdict/$file"|wordsort
else
whilereadword
do
mkwsig"$word"
printf"\t%s\n""$word"
done<"$sysdict/$file"
fi>"$sysdict/$file.anag"
}
done
PlayingwithMatches
Whetherlookingforwordsthatfitapatterninagridorthelettersin
aword,matchingwordsthatfitvariouscriteriaisveryhelpfulin
solvingcrosswordpuzzlesandotherwordgames.It’snotvery
sportingtousetheanagramscripttosolvetheDailyJumbleinthe
newspaper,butifyou’restumpedonthelastword,whynot?
Thesescriptsuseaconfigurationfile,wordfinder.cfg,tofind
thelocationofthewordlists.Ifthefiledoesn’texist,theuserwill
bepromptedtocreateit.Aswithallotheruserconfigurationfilesin
thisbook,itisstoredin$HOME/.config.
Sincethesescriptsusegrep,thefullpowerofregular
expressionsisavailabletotheuser.Thisbookisnottheplaceto
explainthem,butyoucanfindalistofresourcesintheAppendix.
4.8wf—FindWordsThatMatchaPattern
Givenapatternoflettersinacrosswordgrid,forexample,e.r...e,
wherethedotsrepresentemptysquares,Ineedalistofallthewords
thatfitthatpattern.
MyoriginalwfcommandwaswritteninC,anddidnothaveall
thecapabilitiesofthisverysimpleshellscript.
Usage
wf[-c]PATTERN
Besidesusingdotstorepresentunknownletters,thesearchcan
belimitedtoagivensetoflettersinoneormorepositions.For
example,wf....a.f,givesthis:
bedwarf
distaff
engraff
restaff
Ifyouwanttolimittheoutputtowordsthatbeginwitha,d,or
e,youcanreplaceadotwiththoselettersinbrackets:
$wf[ade]...a.f
distaff
engraff
Adotcanalsobereplacedbyarangeofletters;ifyouwanta
searchtomatchonlytheletterscton,youcandothis:
$wf[c-n].e.n.|column
cleansdeeingeternefeeingFresnoGreenehaeingjeeing
creantduennaexeuntfoehnsgeeinggreenshieinglierne
cueingdyeingeyeingfrennegleansgreenyhoeingmoeing
Withthe-coption,wfwillsearchthefileofcompoundwords,
Compounds,andsinglewords.
TheScript
.wf-funcs-sh
description="Findwordsthatmatcha
pattern"
verbose=0
version="1.0"
progname=${0##*/}
wf_cfg=${wf_cfg:$HOME/.config/wordfinder.cfg}
compounds=
whilegetoptsvVcvar
do
case$varin
compounds=1;;
V)version;exit;;
*)printf"%s:invalidoption:%s\n""$progname"
"$var"
exit
;;
esac
done
shift$(($OPTIND-1))
do_config||exit5
compounds=${compounds:+$dict/Compounds}
mask=$1
{
cat$dict/singlewords
if[-n"$compounds"]
then
cut-f1$dict/Compounds
fi
}|grep-i"^$mask$"|sort-fu
4.9wfb—FindWordsThatBeginwithaGiven
Pattern
Givemealistofwordsthatbeginwithagivensetofletters.
HowItWorks
Anothersimplescript,wfbsearchessinglewords,andoptionallythe
Compoundsfileaswell.
Usage
wfb[-c]PATTERN
Aswithwf,thePATTERNcancontaindotsandrangesofletters:
$wfb[a-e]rn|column
arnaarnicasarnuternederning
ArnautArnoarnutserneserns
ArneArnoldBrnoErnestErnst
ArnhemarnottoernErnestine
arnicaarnottoserneErnie
$wfb.ff[j-n]|column
afflationafflictiveaffluxionseffluence
afflationsafflictseffleurageeffluences
afflatusaffluenceeffleurageseffluent
afflatusesaffluenteffloresceeffluents
afflictaffluentlyefflorescedeffluvia
afflictedaffluentnessefflorescenceeffluvial
afflictingaffluentsefflorescenceseffluvium
afflictingsaffluxefflorescenteffluviums
afflictionaffluxesefflorescesefflux
afflictionsaffluxionefflorescingeffluxes
TheScript
description="Findwordsbeginning
withpatternsuppliedonthecommandline"
.wf-funcs-sh
version="1.0"
progname=${0##*/}
whilegetoptsvVcvar
do
case"$var"in
c)compounds=1;;
V)version;exit;;
*);;
esac
done
shift$(($OPTIND-1))
do_config||exit5
{
if[-n"$compounds"]
then
cut-f1$dict/Compounds
fi
}|grep-i"^$1."|sort-fu
4.10wfe—FindWordsThatEndwithaGiven
Pattern
“Forthelifeofme,Icannotcomeupwiththethreecommon
Englishwordsthatendingry.Whatarethey?”
“Isthereawordthatrhymeswithorange?”
HowItWorks
Unfortunately,thereisnoprogramthatwillfindananswertothose
questions.Youcantryitwithwfe,but,unlessyouhaveanunusual
wordlist,youareoutofluck.
Ontheotherhand,wfewillfind31wordsintheUKACDthat
endwithease.
Usage
wfe[-c]PATTERN
Besidesselectingwhichletterstolookfor,alltheseutilities
allowyoutospecifywhichlettersnottolookfor.Tofindallwords
thatendinudebutnottude:
$wfe-c[^t]ude|column
alludecrudeexcludeintrudeprude
bludecumLaudeextrudemagnaCumLaudeQuaalu
Buxtehudedeludeexudeobtrudeschade
choralePreludedenudeGertrudeoccludeseclud
claudedetrudeilludepostludesemiNu
colludedisilludeincludeprecludesubtru
concludeeludeintercludepreludesummaC
coudeemeraudeinterludeprotrudetransu
TheScript
description="Listwordsendingwith
PATTERN"
.wf-funcs-sh
version="1.0"
progname=${0##*/}
compounds=
whilegetoptsvVcvar
do
case$varin
c)compounds=1;;
V)version;exit;;
esac
done
shift$(($OPTIND-1))
do_config||exit5
{
if[-n"$compounds"]
then
cut-f1"$compounds"
fi
}|grep-i".$1$"|sort-fu
4.11wfc—FindWordsThatContainaGiven
Pattern
“Iwouldlikealistofallthewordsthatcontainthewordfast.”
HowItWorks
Thisisasimplejobforgrep.Putitinascriptsoyouwouldn’thave
torememberwhichdirectorycontainsthewordlistsandwhatthe
filenamesare.
Usage
wfc[-c]PATTERN
Tofindwordsthatcontainfast:
$wfcfast|column
breakfastedhandfastssteadfastnessunfastidious
breakfastingheadfastsstedfastsunsteadfastly
breakfastsmakefastsunfastenunsteadfastnes
handfastedshamefastnessunfastened
handfastingsitfastsunfastening
handfastingssteadfastlyunfastens
Byaddingthe-coption,youcanexpandthesearchtoinclude
compoundwordsandphrases:
$wfc-cfast|column
aFastBucklifeInTheFastLane
BreakfastAtTiffanySmakefasts
breakfastedplayFastAndLoose
breakfastingpowerBreakfasts
breakfastRoompullAFastOne
breakfastsshamefastness
breakfastSetsitfasts
breakfastTablesnapFastener
breakfastTablessnapFasteners
breakfastTVsteadfastly
continentalBreakfastssteadfastness
copperFastenstedfasts
copperFastenedunfasten
copperFasteningunfastened
copperFastensunfastening
goFasterStripesunfastens
handfastedunfastidious
handfastingunsteadfastly
handfastingsunsteadfastness
handfastsweddingBreakfasts
headfastsyouCanTGetBloodOutOfAStone
heTravelsFastestWhoTravelsAlone
zipFastener
inTheFastLanezipFasteners
TheScript
description="Findwordscontaining
thepatterngivenonthecommandline"
version="1.0"
progname=${0##*/}
compounds=
opts=c
do
case$varin
c)compounds=1;;
*)exit;;
esac
done
shift$(($OPTIND-1))
.wf-funcs
pattern=$1
{
if[-n"$compounds"]
then
cut-f1"$compounds"
fi
}|grep-i".$pattern."|sort-fu
Notes
Wordswillnotmatchifthepatternisatthebeginningorendofthe
word.Ifyouwantthosetomatch,changethelastlineofthescript
tothis:
}|grep-i"$pattern"|sort-fu
4.12wfit—FindWordsThatFitTogetherina
Grid
Ifyouneedtofindwordsthatwillfitinthetopcornerofthegrid
(seeFigure4-2).Thethirdletterin1-DOWNmustfitintoaword
thatgoesacross.Ifyouputpartinthere,therewillbenothingthat
cancompletetheintersectingword.
Figure4-2. Apartiallycompletedcrosswordgrid
HowItWorks
Specifythepatternforeachword,followedbythepositioninthe
wordoftheintersectingletter,asargumentstowfit.
Usage
wfitPATTERN1POS1PATTERN2POS2
ThecommandfortheexampleinthegridshowninFigure4-2is
this:
$wfitp..t3..m.t.c1
pact-cometic
Pict
pant-nematic
pent
pint
pont
punt
phot-osmotic
plot
poot
pyot
past-sematic
pestSemitic
postsomatic
psstsomitic
TheScript
##requiresbash2+orksh93
description="Findwordsthatfit
togetherinagrid"
version="1.0"
progname=${0##*/}
compounds=
whilegetoptsvVcvar
do
case$varin
c)compounds=1;;
V)version;exit;;
*);;
esac
done
shift$(($OPTIND-1))
max(){
_MAX=$1
_MIN=$1
shift
fornum
do
[$num-gt$_MAX]&&_MAX=$num
[$num-lt$_MIN]&&_MIN=$num
done
}
##failiftherearefewerthan4
arguments
[$#-ne4]&&exit5
mask1=$1
mask2=$3
ltr1=$2
ltr2=$4
IFS=$'\t\n'
list1=(`wf${compounds:+-c}
"$mask1"`)
letters=`printf"%s\n""${list1[@]}"
|cut-c$ltr1|sort-fu`
forletterin$letters
do
maskA=${mask1:0:ltr1-1}$letter${mask1:ltr1}
maskB=${mask2:0:ltr2-1}$letter${mask2:ltr2}
list1=(`wf${compounds:+-c}"$maskA"`)
list2=(`wf${compounds:+-c}"$maskB"`)
[${#list2}-eq0]&&continue
max${#list1[@]}${#list2[@]}
n=0
w1=${#list1[0]}
w2=${#list2[0]}
[$verbose-ge1]&&echo"MAX=$_MAXMIN=$_MIN
maskA=$maskAmaskB=$maskB"
while[$n-lt$_MAX]
do
if["${#list2[$n]}"-gt0]
then
if["${#list1[$n]}"-gt0-a$n-eq0]
then
printf"%${w1}s-%${w2}s\n""${list1[n]}"
"${list2[$n]}"
else
printf"%${w1}s%${w2}s\n""${list1[n]}"
"${list2[$n]}"
fi
else
printf"%${w1}s\n""${list1[n]}"
fi
n=$(($n+1))
done
echo
done
Notes
Thisisoneofthefewscriptsinthebookthatrequiresbash2+or
ksh93,asitusesarraysandsubstringparameterexpansion.
4.13anagram—FindWordsThatAreAnagrams
ofaGivenWord
“IhavethelettersADGETGleftonmyScrabblerack;whatsixletterwordscanImakewiththem?”
HowItWorks
Ifyouranthewf-setupscript,youshouldhaveafilethathaseach
wordpairedwithitssignature.Sortthelettersonyourrack
alphabeticallytocreateasignature(usingthemkwsigfunction)and
findwordswiththatsignatureinthesinglewords.anagfile.Orlet
theanagramscriptdoitforyou.
Usage
anagram[-c]WORD
Ifyouusethe-coption,anagramwillsearchboth
singlewords.anagandCompounds.anag:
$anagram-shoutcast
outacts
outcast
$anagram-sh-coutcast
actsOut
castOut
outacts
outcast
TheScript
description="Findwordsthatare
anagramsofWORDoncommandline"
version="1.0"
progname=${0##*/}
compounds=
.wf-funcs
whilegetoptsVcvar
do
case$varin
c)compounds=1;;
V)printf"%s:%s\n"$progname$version;exit;;
esac
done
shift$(($OPTIND-1))
do_config||exit5
pattern=`mkwsig$1`
compounds=${compounds:+$dict/Compounds.anag}
grep-i"^$pattern$TAB"
$dict/singlewords.anag$compounds|
cut-f2|
sort-fu
4.14aplus—FindAnagramsofaWordwitha
LetterAdded
“Iamtryingtosolveacrypticcrosswordclue.Iknowtheansweris
ananagramofSATIREwithN,E,W,orSadded.HowcanIgeta
listofallthepossiblewords?”
HowItWorks
Youcouldusefourcommands:anagramsatiren,anagramsatiree,
anagramsatirew,andanagramsatires.However,whynotletthe
computerdotheworkwithaplus?Thiscommandwillfind
anagramsofthewordwitheachletterofthealphabetaddedinturn.
Usage
aplus[-c][-l]WORD
Withthe-loption,apluswilljustaddthelettersyoutellitto
use.The-coptionincludescompoundwords:
$aplusnearly
+B:blarney
+C:larceny
+G:angerly
+I:inlayernailery
+P:plenary
$aplus-c-lbognearly
+b:blarney
+o:earlyOn
+g:angerly
TheScript
progname=${0##*/}
compounds=
letters=ABCDEFGHIJKLMNOPQRSTUVWXYZ
whilegetoptscl:var
do
case$varin
c)compounds=1;;
l)letters=$OPTARG;;
esac
done
shift$(($OPTIND-1))
word=$1
while[-n"$letters"]
do
_temp=${letters#?}
l=${letters%$_temp}
letters=$_temp
printf"+$l:\r">&2
result=ànagram${compounds:+-c}"$word$l"`
[-n"$result"]&&{
printf"+%s:""$l"
echo$result
}
done
4.15aminus—RemoveEachLetterinTurnand
AnagramWhat’sLeft
ManyofthecrypticcrosswordcluesChriswritesuseanagrams,
sometimesofthewholeword,sometimesofpartoftheword.
Amongotherthings,heneedaprogramtolisttheanagramsofa
wordwhenlettersareremoved.
HowItWorks
Theaminusscriptremoveseachletterinturn(butdoesnotremovea
lettermorethanonceifthereareduplicates),thenfindsthe
anagramsofwhat’sleft.
Usage
aminus[-c]WORD
Withoutthe-coptionthatincludescompoundwordsinthe
result,aminuswouldfindonlyanchoritesandantechoirswhenfed
neoChristianity:
$aminus-sh-cneoChristianity
-i:ancientHistory
-y:anchoritesantechoirschainStore
TheScript
version="1.0"
progname=${0##*/}
compounds=
whilegetoptscvar
do
case$varin
c)compounds=1;;
esac
done
shift$(($OPTIND-1))
word=$1
done=
left=
right=$word
while[-n"$right"]
do
temp=${right#?}
l=${right%$temp}
##checkwhetherthesameletterhasalreadybeen
removed
case$leftin
*$l*)right=$temp
continue;;##Don’trepeataletter
esac
result=ànagram${compounds:+-c}
$left$temp`
[-n"$result"]&&{
printf"%c%s:""-""$l"
echo$result
}
left=$left${l}
right=$temp
done|sort
Summary
FromcomposingcrosswordpuzzlestosolvingtheDailyJumblein
yourlocalnewspaper,thescriptsinthischaptercanhelpyou.Ifyou
wanttoknowwhethertherewasaseven-letterwordontherackyou
hadinthatScrabblegame,orwhetheryoucouldhavesqueezeda
wordinto_A_N__E__S,thetoolsarehere.
Someofthesescriptsareofmostusetocruciverbalists,but
manycanbeusedbyanyonewhoplayswordgames.
(1) Ontario,Canada
Numbercrunchingwasoncethedomainofbigmainframe
computershiddenawayinsecure,air-conditionedrooms.Now,the
computeronyourdesk(oreventheoneinyourpocket)hasmore
computingpowerthanthoseoldbehemoths.Theshell,withthehelp
ofexternalcommands,canharnessthatpower.
POSIXshellshavearithmeticbuiltin,butarelimitedto
integers.Multiplicationofdecimalfractionscanbedoneintheshell
withthefpmulfunction,whichappearsinthemath-funcslibrary
describednext.Otheroperationscouldalsobecodedwithshell
arithmetic,buttheyaremorecomplicated,andexternalcommands
suchasawkcandothejobasefficiently.Similarly,thereisnopoint
tousingashellfunctiontomultiplyiftheresultisthengoingtobe
passedtoawkforadditionorsubtraction;letawkhandleitall.
Thescriptsinthischapterareallrelativelysimple,butuseful
nonetheless.Themajoritycanbeusedatthecommandlineorin
anotherscript.“Numbercrunching”isarathergrandtermforthe
scriptsinthischapter,buttheycanbethebasisforcomplex
calculations.
Themath-funcsLibrary
Themath-funcslibrarycontainsanumberofshellfunctionsthat
willbeusedinmanyscripts.Somearealsousefulatthecommand
line.Forinstance,Chrisusescalc,asimplecommand-line
calculator,almosteveryday.
Mostofthesefunctionscomeinpairs.Onebeginswithan
underscoreandsetsavariable.Thatvariableisthenameofthe
functionconvertedtocapitalletters:_roundsetsthevariable_ROUND.
Thesecondfunctioncallstheunderscoreversionandprintsthe
result.
Whytwofunctions?Inallshellsexceptksh93,command
substitutionforksanewprocess.Inotherwords,command
substitutionisslow.Itgreatlydiminishestheadvantageofusinga
functioninsteadofanexternalscript.Thedifferenceisnot
significantifitiscalledonlyonce,butinascriptwherethefunction
isusedmanytimesorwheremanyfunctionsarecalled,itcanmake
thedifferencebetweenaquick,responsivescriptandasluggishone.
Tousetheunderscorefunction,callit,theneitherusethe
variablethathasbeenset(astheargumenttoanothercommand),or
assignittoanothervariable:
_funcarg1arg1
printf"Theansweris:%s\n""$_FUNC"
funky=$_FUNC
Itismuchfasterthancommandsubstitution:
printf"Theansweris:%s\n""`funcarg1
arg2`"
funky=`funcarg1arg2`
Generally,whenyouwanttoprinttheresult(asyouwouldat
thecommandline),usetheplainfunction.Otherwise,usethe
underscoreversion.
5.1fpmul—MultiplyDecimalFractions
Mostshellscannotdofloating-pointarithmeticinternally;an
externalprogramsuchasbcorawkmustbeused.Isthereafaster
way?
HowItWorks
Inmanycases,awkandbcarethebestwaytoperformcalculations
withfractions,butmultiplication,onitsown,canbedonewitha
functionmorequicklythanwithanexternalcommand.
Usage
fpmul
N[.N]N[.N][...]
_fpmulN[.N]N[.N][...]##the
resultisin$_FPMUL
Nomultiplicationsignisusedbetweentheoperands,although
youcanchangealineinthefunctiontoallowthem,andothernonnumericelements,tobeinterspersedonthecommandlineand
ignored.Normally,thecommandisinvokedwithtwoormore
operands:
$fpmul12.34567.89
838.10205
Inascript,youwouldusetheunderscoreversion,whichsetsa
variableratherthanprinttheanswer:
$_fpmul1.231.111.011.21.231.11
1.011.21.231.1.2.4
$echo$_FPMUL
0.2963802115596415104
Noticethelengthofthefraction.Onsomeshells,thatwillcause
problems:
$fpmul1.231.111.011.21.231.11
1.011.21.231.1.2.4
fpmul:overflowerror:-1711122980
Sincefpmulusestheshell’sbuilt-inarithmeticcapabilities,itis
constrainedbyitsinternallimits.Ashell’sintegersizeisusually32
bits,whichmeansthelargestnumberitcanhandleis2,147,483,647.
Versionsofbashlaterthan2.05a,whencompiledonasystemthat
canhandlethem,uses64-bitintegersandhasamaximumintegerof
9,223,372,036,854,775,807.However,themethodformultiplying
decimalfractionscomplicatesmatters.
Inschool,moredecadesagothanIcaretoadmit,Iwastaughta
methodofcalculatingtheproductofdecimalfractions:removethe
decimalpointsandmultiplytheresultingnumbers.Getthetotal
numberofdecimalplacesintheoriginalfractions,andinsertthe
decimalpointthatmanyplacesfromtheend.Forexample,to
multiply1.5by1.2,youwouldmultiply15by12(180),andinsert
thedecimalpoint2placesfromtheend,giving1.80.Thefpmul
functionremovestrailingzeros,andtheansweris1.8.
Althoughthefinalanswerwaslessthan2,thecalculationuseda
numberashighas180.Thehighestnumberusedwillgrowasthe
numberofdecimaldigitsgrows,ratherthanasthevalueofthe
resultincreases.Youcanseethisbyinvokingthe“showyourwork
totheteacher”clauseincludedinthefunction;itcanbeinvokedby
settingverboseto63.
Withonlyafewdecimalplaces,wecanseethattheintegers
remainwellwithinthelimits:
$verbose=63fpmul3.144.5
fp_tot=$((1*314))
fp_tot=$((314*45))
fp_tot=14130
14.13
Whenthenumberincreases,eventhougheachoperandhasonly
twodigits,theintegersbeingusedinthecalculationsgrowquickly.
Thiscalculationsucceedsinbashwith64-bitintegers,butfailsin
ashwithonly32bits:
bash2.05b:$verbose=63fpmul1.2
3.45.67.89.87.65.4
fp_tot=$((1*12))
fp_tot=$((12*34))
fp_tot=$((408*56))
fp_tot=$((22848*78))
fp_tot=$((1782144*98))
fp_tot=$((174650112*76))
fp_tot=$((13273408512*54))
fp_tot=716764059648
71676.4059648
ash:$fpmul1.23.45.67.89.87.6
5.4
fpmul:overflowerror:-495478784
TheScript
.standard-funcs
_fpmul()
{
fp_places=
fp_tot=1
fp_qm=
fp_neg=
_FPMUL=
forfp_n
do
##2negativesmakeapositive,i.e.,each
negativenumberchangesthesign
case$fp_nin
-*)["$fp_neg"='-']&&fp_neg=||
fp_neg='-'
fp_n=${fp_n#-}
;;
esac
##Checkfornon-numericcharacters
case$fp_nin
##(minussignshavebeenremovedbythis
point,
##soweDon’tneedtoincludethem)
*[!0-9.]*)return1;;
##Usethisinplaceoftheline
aboveifyouprefertoignore(i.e.,skipover)
##invalidarguments
##*[!0-9.]*)continue;;
esac
##countthenumberofdecimalplaces,
##thenremovethedecimalpointandmultiply
case$fp_nin
.*)fp_int=
fp_dec=${fp_n#?}
fp_places=$fp_places$fp_dec
fp_n=$fp_dec
;;
*.*)fp_dec=${fp_n#*.}
fp_int=${fp_n%.*}
fp_places=$fp_places$fp_dec
fp_n=$fp_int$fp_dec
;;
esac
##removeleadingzeroes
while:
do
case$fp_nin
0*)fp_n=${fp_n#0};;
*)break;;
esac
done
##"Showyourworktotheteacher"ifverbose
equals63
[${verbose:-0}-eq63]&&printf"%s\n"
"total=\$(($fp_tot*$fp_n))"
##multiplybytheprevious
totalfp_tot=$(($fp_tot*$fp_n))
##reportanyoverflowerror
case$fp_totin
-*)printf"fpmul:overflowerror:%s\n"
"$fp_tot">&2
return1
;;
esac
done
[${verbose:-0}-eq63]&&printf"%s\n"
"total=$fp_tot"
##padtheresultwithleadingzeroesifnecessary
while[${#fp_tot}-lt${#fp_places}]
do
fp_tot=0$fp_tot
done
fp_df=
while[${#fp_df}-lt${#fp_places}]
do
left=${fp_tot%?}
fp_df=${fp_tot#$left}$fp_df
fp_tot=$left
done
_FPMUL=$fp_tot${fp_df:+.$fp_df}
##removetrailingzeroesordecimalpoints
while:
do
case$_FPMULin
*.*[0\]|*.)_FPMUL=${_FPMUL%?};;
.*)_FPMUL=0$_FPMUL;;
*)break;;
esac
done
}
fpmul()
{
_fpmul"$@"||return1
printf"%s\n""$_FPMUL"
}
Notes
Thefpmulfunctioncouldbeimprovedinvariousways:
Usingavariableforthedecimalpointwouldallowthescriptto
beadjustedforuseinlocaleswhereacomma(orother
character)isusedinsteadofaperiod.
Insteadoffailing,thefunctioncouldcallawkorbcwhenthere
isanoverflow.
Theargumentscouldbecheckedforinvalidnumbers.
Floating-pointarithmeticisbuiltintoksh93,sofpmulis
unnecessaryinsuchscripts.ThisalternativeprovidesaKornShell
drop-infunctionthatiscompatiblewithallscriptsthatusefpmul.
_fpmul()
{
_FPMUL=1
forfp_n
do
_FPMUL=$(($_FPMUL*$fp_n))
done
}
5.2int—ReturntheIntegerPortionofaDecimal
Fraction
Toconvertadecimalfractiontoanintegerinascript,thescriptwill
oftenputtheexpansionin-line,ratherthancallafunction,butatthe
commandlineyoucoulduseanintcommand.
HowItWorks
Simpleparameterexpansionisallittakestoreturntheportionofa
numberthatprecedesthedecimalpoint.
Usage
_intN[.N]##theresultisin$_INT
int
N[.N]
TheScript
_int()
{
_INT=${1%%.*}
}
int()
{
printf"%d\n""${1%%.*}"
}
5.3round—RoundtheArgumenttotheNearest
Integer
Alittlemorecomplexthanint,roundtakesthenumbertothe
nearestinteger.Forexample,2.3roundsto2,but2.7roundsupto3.
HowItWorks
Usingparameterexpansion,thenumberissplitintoitsintegerand
fractionalparts.Ifthefractionalpartislessthan.5,roundreturnsthe
integerpart;otherwise,thenexthigherintegerisreturned.
Usage
_round
N[.N]##theresultisin$_ROUND
roundN[.N]
TheScript
_round()
{
_ROUND=${1%%.*}##extracttheinteger
case${1#*.}in
##Ifthedecimalbeginswithanydigitfrom
fiveto9,
##theresultisroundedup
[5-9]*)_ROUND=$(($_ROUND+1));;
esac
}
round()
{
_round"$1"
printf"%s\n"$_ROUND
}
5.4pow—RaiseaNumbertoAnyGivenPower
Fromfindingtheareaofasquaretoturningscientificnotationintoa
human-readablenumber,exponentiation(raisinganumbertoa
givenpower)isafundamentalmathematicaloperation.Whilethisis
easilyhandledbyexternalutilitiessuchasawk,youwouldprefera
moreefficientmethodinmostcases.
HowItWorks
Iftheexponentrequestedisadecimalfraction,I’mnotevengoing
toattempttodothecalculationintheshell;I’lljustpassitalongto
awk.Iftheexponentisaninteger,butthebasecontainsadecimal
fraction,fpmulisneeded.Ifboththebaseandtheexponentare
integers,shellarithmetichaswhatittakestoproducetheanswer.
Usage
_pow
BASEEXPONENT##Theresultisin
$_POW
powBASEEXPONENT##Theresultis
printed
Thebase,theexponent,orbothcanbeeitheranintegerora
decimalfraction.Thesearepassed,basefirst,withoutany
interveningoperator:
$pow124
20736
TheScript
_pow()
{
exp=$2
pow_arg=
case$expin
##Iftheexponentcontainsadecimal,
##We’llwimpoutandletawkhandleit
*.*)_POW=$(awk"BEGIN{printf\"%f\",$1**
$2;exit}")
return;;
##Forsecondandthirdpowers,wepassthe
calculationtothemore
##efficientfunctionsdefinedbelow
2)_square"$@"
_POW=$_SQUARE
return
;;
3)_cube"$@"
_POW=$_CUBE
return
;;
esac
##Ifthebasecontainsnodecimalpoint,wecanuse
shellarithmetic
case$1in
*.*)pow_op=;;
*)pow_op="*";;
esac
##Buildthestringthatwillbeusedin$((...))
##orastheargumentsto_fpmul
while[$exp-gt0]
do
pow_arg="${pow_arg:+$pow_arg$pow_op}$1"
exp=$(($exp-1))
done
case$pow_argin
*.*)_fpmul$pow_arg##Decimalpointsfound
_POW=$_FPMUL
;;
*)_POW=$(($pow_arg));;
esac
}
pow()
{
_pow"$@"&&printf"%s\n""$_POW"
}
5.5square—RaiseaNumbertotheSecond
Power
Referto5.6,cube,whichworksthesameway.Bothscriptsare
describedthere.
5.6cube—RaiseaNumbertotheThirdPower
Raisinganumbertothesecondorthirdpowerarespecialinstances
ofpow.Theyareusedoftenenoughtowarranttheirownfunctions,
andtheycanbedealtwithinasimplerfashion.
HowItWorks
Ifthenumbercontainsadecimalpoint,use_fpmul;otherwise,use
shellarithmetic.(Notethatweuse_fpmul,notfpmul,asthere’sno
needtoprinttheresult;thatwillbedonebysquareorcube,if
needed.)
Usage
_square
N[.N]##theresultisstoredin
$_SQUARE
square[N[.N]]
_cube
N[.N]##theresultisstoredin
$_CUBE
cube[N[.N]]
TheScript
_square()
{
case$1in
*.*)_fpmul$1$1##thebase
containsadecimalpoint
_SQUARE=$_FPMUL
;;
*)_SQUARE=$(($1*$1));;##no
decimalpoint;useshellarithmetic
esac
}
square()
{
arg"$@"##Theargfunctionwasintroduced
inChapter
1
_square"$arg"
printf"%s\n""$_SQUARE"
}
_cube()
{
case$1in
*.*)_fpmul$1$1$1
_CUBE=$_FPMUL
;;
*)_CUBE=$(($1*$1*$1));;
esac
}
cube()
{
arg"$@"
_cube"$arg"
printf"%s\n""$_CUBE"
}
5.7calc—ASimpleCommand-LineCalculator
Frommechanicalcalculatorsandsliderules,topocketcalculators
andfinallycomputers,devicesforrelievingthementalstrainof
solvingmathematicalproblemshavebeenwithusforcenturies.
Everymoderncomputercomeswithatleastoneprogramthat
presentsanon-screenversionofapocketcalculator.I’vetriedthem,
butIfindthemslowtoloadandcumbersometouse.Whatanyone
wouldwantissomethingthatallowsyoutotypein“23*45”and
givesyou1,035withoutanyfuss.
HowItWorks
Theawkprogramminglanguagecandoanycalculationoneislikely
toneed.Allthatisneededisaninterfacetoit.Thecalcscript,
whichcouldbeinstalledasafunction,simplyincorporatesthe
expressioninanawkprintstatement.
Usage
calc"EXPRESSION"
EXPRESSIONcanbeanyexpressionawkcanunderstand.This
includesusingfunctionssuchassqrt:
$calc"sqrt(666)"
25.807
Moreusualisarun-of-the-millarithmeticexpression:
$calc"23.45*11+65"
322.95
Theexponentiationoperatorcanbeeitheradoubleasterisk(as
seeninthesquareandcubefunctionsearlierinthechapter)orthe
caret:
$calc'23^3'
12167
Charactersthatarespecialtotheshellmustbequotedor
escaped.Itisrecommendedtoalwaysenclosetheentireexpression
inquotes.
TheScript
calc()
{
awk'BEGIN{print'"$*"';exit}'
}
AddingandAveraging
Thesescriptsallworkonalistofnumbersobtainedfromoneor
morefiles,orfromthestandardinput.Theyincludethefourmain
typesofaverages,namelythemean,median,mode,andrange.The
first,total,isthesimplest;itjustaddsthemupandprintstheresult.
Themostcomplexisstdev,whichprintsthestandarddeviationofa
setofnumbersgiveninlistortableformat.
5.8total—AddaListofNumbers
Sayyouhavealistofnumbersstoredinafile,onetoaline.Now
youwanttoaddthemalltogether.
HowItWorks
Ifthesenumberswereallguaranteedtobeintegers,thiscouldbe
donewithshellarithmetic(seethenotes).Byusingawk,decimal
fractionscanalsobeaccommodated.
Usage
total
[FILE...]
Ifthenumbersinthefilearenotonetoaline,autilitysuchas
prw(fromChapter2)canbeused:
prwFILE|total
Thenumberofdecimalplacesintheoutputcanbecontrolled
withtheprecisionvariable.Theprecisionisexpressedasaprintf
widthmodifier:
$printf"%s\n"2.34534.567871.0055
|precision=.2total
37.92
Thevalue.2indicatedthatthenumberwastobeprintedwith
twodigitsafterthedecimalpoint.Anumberbeforethedecimal
pointintheprecisionvariabledenotesthetotalwidthinwhichthe
resultistobeprinted,andprintsitflushrightinthatspace:
$printf"%s\n"2.34534.567871.0055
|precision=19.3total
37.918
Thenumberendedatthe19thcolumn,andusedthreedecimal
digits.
TheScript
awk`##Addthevalueofeachline
tothetotal
{total+=$1}
##Afterallthelineshavebeenprocessed,
printtheresult
END{printf"%"prec"f\n",total}'prec=$precision
${1+"$@"}
Notes
Ifallthenumbersareintegers,thisscriptwillworkevenwhenthere
aremultiplenumbersonaline:
echo$((`cat${1+"$@"}|tr-s'\012'
'++'`0))
5.9mean—FindtheArithmeticMeanofaList
ofNumbers
Ateachercalculatingastudent’sgradeaveragemustfindthe
arithmeticmean,orthesumofallthenumbersdividedbythe
numberofnumbers.
HowItWorks
Themeanscripttakesnumbers,onetoaline,fromoneormorefiles,
orfromthestandardinput,andprintsthearithmeticmean.Thereis
alsoanoptiontoreadtheinputasatable,eachlinecomprisingthe
numberanditsnumberofoccurrences.
Usage
mean[-t|-T][FILE...]
Thesimplestuseistoaveragealistofnumbersinafile:
$catnumlist
3
11
4
8
7
9
6
11
7
8
$meannumlist
7.4
Toaveragealistenteredatthecommandline,usethis:
$printf"%s\n"2546752197213
674587674532|mean
37.8667
Let’sconsideranotherexample.IfIrolledapairofdiceten
times,theresultscouldbeexpressedasatable:
Table4-1. TenRollsofTwoDice
Number NumberofTimesRolled
1
1
3
2
6
1
7
4
10
1
12
1
Thistablecanbepipedthroughmeanwiththe-toption:
$echo"11
32
61
74
101
121"|mean-t
6.3
Toaccommodatetablesgeneratedbyuniq-c,whichhavethe
frequencybeforethedataitem,thereisthe-Toption:
$throw10|sort|uniq-c|mean-T
7.4
$throw10|sort|uniq-c|mean-T
6.9
(Thethrowcommand,whichappearslaterinChapter18,
simulatestherollingoftwosix-sideddicentimes.)
TheScript
table=0
##Parsecommandlineotions
whilegetoptsVtTvar
do
case$varin
t)table=1;;
T)table=2;;
V)echo$version;exit;;
esac
done
shift$(($OPTIND-1))
##Therearethreedifferentawk
programs,oneforeachinputformat
if[$table-eq0]##Onenumberper
line
then
awk'{tot+=$1}##Addnumbertothe
total
END{printtot/NR}##Dividetotalby
numberoflines
'${1+"$@"}
elif[$table-eq1]##Eachline
hasnumberofoccurrencesfollowedbythenumber
then
awk'{
num+=$2##Counttotalnumberof
occurrences
tot+=$1*$2##Totalisnumbermultipliedby
numberofoccurrences
}
END{printtot/num}'${1+"$@"}##Divide
totalbynumberofoccurrences
elif[$table-eq2]##Eachline
hasnumberfollowedbythenumberofoccurrences
then
awk'{
num+=$1##Counttotalnumberof
occurrences
tot+=$1*$2##Totalisnumbermultiplied
bynumberofoccurrences
}
END{printtot/num}'${1+"$@"}##Divide
totalbynumberofoccurrences
fi
5.10median—FindtheMedianofaListof
Numbers
Ateacherwantingtoknowhowwelltheclassasawholehasdone
onatestmayusethemeantogettheaverageresult,orusethe
median,whichisthemidpoint—thenumberinthemiddlethathas
thesamenumberofitemsaboveandbelowit.
HowItWorks
Whenalisthasanoddnumberofitems,findingthemedianiseasy;
wheretherearennumbers,sortthem,andthemedianisonline(n+
1)/2.It’saneatone-liner:
sed-n"$(((`sort-n${1+"$@"}|tee
$tmp|wc-l`+1)/2)){p;q;}"$tmp
Inasetwithanevennumberofelements,themedianisthe
arithmeticmeanofthetwocentralelements.Forthat,theeasiest
wayistopipethesortedlistthroughanawkscript.Ofcourse,italso
worksforanoddorevennumberofelements.
Usage
median
[FILE...]
Tofindthemedianofthenumlistfile(usedinthepreviousmean
example),thecommandis
$mediannumlist
7.5
Let’sconsideranotherexample.Themedianof20rollsofapair
ofdicecanbecalculatedusingthefollowing:
$throw20|median
7
TheScript
progname=${0##*/}
##Sortthelistobtainedfromoneor
morefilesorthestandardinput
sort-n${1+"$@"}|
awk'{x[NR]=$1}##Storeallthevaluesinan
array
END{
##Findthemiddlenumber
num=int((NR+1)/2)
##Ifthereareanoddnumberofvalues
##usethemiddlenumber
if(NR%2==1)printx[num]
##otherwiseaveragethetwomiddlenumbers
elseprint(x[num]+x[num+1])/2
}’
5.11mode—FindtheNumberThatAppears
MostinaList
Havinggradedallthetests,theteacherwantstoknowwhatgrade
themoststudentsachieved.Thisisthemode—themostfrequently
occurringvalueinalist.
HowItWorks
Youcanderiveanapproximateanswerbyusingsortanduniq-c
(andthatmaybeadequateforyourpurposes),butthemodemay
containnone,one,ormorethanonevalue.Ifnovalueappearsmore
thanonceinthedataset,thereisnomode;ifmorethanonevalue
appearswiththesamefrequencythatishigherthanalltheothers,
theyareallmodalvaluesofthatsetofnumbers.
Usage
mode
[-t|-T][FILE...]
Theteachercanpipeallthegradesthroughmodetogetthe
answer:
$printf"%s\n"ABABBCDFCBA
CDCFCBDFCCBDCFD|mode
C
Inthenumlistfile(usedasanexamplewiththemeanscript),
threenumbershavetwoinstances(andnonehavemorethantwo);
therefore,modeprintsallthree:
$modenumlist
11
7
8
Aswiththemeanscript,the-tand-Toptionsacceptthedatain
tableformat.
TheScript
progname=${0##*/}
table=0##bydefaultthe
filecontainsasimplelist,notatable
##Parsethecommand-lineoptions
whilegetoptstTvar
do
case$varin
t)table=1;;
T)table=2;;
esac
done
shift$(($OPTIND-1))
##Therearereallythreeseparatescripts,
##oneforeachconfigurationoftheinputfile
if[$table-eq0]
then
##Theinputisasimplelistinonecolumn
awk'{
++x[$1]##Counttheinstancesofeach
value
if(x[$1]>max)max=x[$1]##Keeptrack
ofwhichhasthemost
}
END{
if(max==1)exit##Thereisnomode
##Printallvalueswithmaxinstances
for(numinx)if(x[num]==max)print
num
}'${1+"$@"}
elif[$table-eq1]
then
##Thesecondcolumncontainsthenumberof
instancesof
##thevalueinthefirstcolumn
awk'{x[$1]+=$2##Addthenumberofinstancesof
eachvalue
if(x[$1]>max)max=x[$1]##Keep
trackofthehighestnumber
}
END{##Printtheresult
if(max==1)exit
for(numinx)if(x[num]==max)
printnum
}'${1+"$@"}
elif[$table-eq2]
then
##Thefirstcolumncontainsthenumberofinstances
of
##thevalueinthesecondcolumn
awk'{x[$2]+=$1
if(x[$1]>max)max=x[$2]
}
END{##Printtheresult
if(max==1)exit
for(numinx)if(x[num]==max)print
num
}'${1+"$@"}
fi|sort
5.12range—FindtheRangeofaSetof
Numbers
Thefourthmaintypeofaverage,aftermean,median,andmode,is
range:thedifferencebetweenthehighestandlowestnumbersinthe
set.
HowItWorks
Intherangecommand,anawkscriptreadstheinputandstoresthe
highestandlowestnumbersuntiltheendofinputisreached.Atthat
point,thedifferencebetweenthetwoisprinted.
Usage
range[-nN][FILE...]
Usingthenumlistfileasinput,therangeis8,thedifference
between11and3:
$rangenumlist
8
We’llstoreasampleinthevariabletable:
$table="112
75
331
24"
Bydefault,rangeusesthefirstcolumn:
$echo"$table"|range
31
The-noptiontellsrangetoanalyzecolumnNoftheinput.This
exampleusescolumn2:
$printf"%s\n""$table"|range-n2
##usescolumn2
4
Becausethevariable,table,isnotquotedinthenextexample,
eachnumberinthetableisprintedonaseparateline,andrange
readsitasasimplelist:
$printf"%s\n"$table|range
32
TheScript
progname=${0##*/}
column=1
whilegetoptsVn:var
do
case$varin
n)column=$OPTARG;;
esac
done
shift$(($OPTIND-1))
awk'BEGIN{
##awkvariablesareinitializedto0,sowe
needto
##giveminavaluelargerthananythingitis
likelyto
##encounter
min=999999999999999999}
{
++x[$col]
if($col>max)max=$col
if($col<min)min=$col
}
END{printmax-min}
'col=$column${1+"$@"}
5.13stdev—FindingtheStandardDeviation
Thestandarddeviationisameasureofthedispersionofasetof
datafromitsmean.Themorespreadapartthedatais,thegreaterthe
deviation.Oneuseofstandarddeviationisinfinance,wherethe
standarddeviationisusedasanindicatorofthevolatilityofastock.
HowItWorks
Oneoftwodifferentawkprogramsisuseddependingonwhetherthe
inputisalistofnumbersoratablewiththefrequencyandthe
numberonthesameline.
Usage
stdev
[-t][FILE...]
Aswithotherscriptsinthissection,stdevcantakeitsdatafrom
oneormorefilesorfromthestandardinput.Thedatacanbeina
list,or,ifyouusethe-toption,itcanbeintheformofatablewith
theitemsinthefirstcolumnandthefrequencyinthesecond.
TheScript
progname=${0##*/}
table=0
precision=.2
whilegetoptstp:var
do
case$varin
t)table=1;;
p)precision=$OPTARG;;
esac
done
shift$(($OPTIND-1))
if[$table-eq0]##Valuesinput
oneperline
then
awk'{
tot+=$1##Addvaluetototal
x[NR]=$1##Addvaluetoarrayforlater
processing
}
END{
mean=tot/NR##Calculate
arithmeticalmean
tot=0##Resettotal
for(numinx){
##Thedifferencebetweeneach
numberandthemean
diff=x[num]–mean
##Squaredifference,multiply
bythefrequency,andaddtototal
tot+=diff*diff
}
##Deviationistotaldividedby
numberofitems
printf"%"precision"f\n",sqrt(
tot/NR)
}'precision=$precision${1+"$@"}
else##
awk'{
num+=$2
tot+=$1*$2
items+=$2
x[NR]=$1
f[NR]=$2
}
END{
mean=tot/num##Calculate
arithmeticalmean
tot=0##Resettotal
for(numinx){
##Thedifferencebetween
eachnumberandthemean
diff=x[num]–mean
##Squaredifference,
multiplybythefrequency,andaddtototal
tot+=diff*diff*
f[num]
}
printf"%"precision"f\n",
sqrt(tot/items)
}'precision=$precision${1+"$@"}
fi
ConvertingBetweenUnitSystems
Is25°warmorcold?Is1,000meterstoofartowalkforaloafof
bread?Whichismore,500gramsor1pound?WillIgetaspeeding
ticketifmyspeedometersays100?
ChrisgrewupusingimperialweightsandmeasuresintheU.K.,
theU.S.,andCanada,andanadultlivinginCanadawhenthe
countrychangedtothemetricsystem.Rather,itstartedtochange,
thenitdescendedintochaos.Thechangehasnotbeenenforced
since1983,andthereisnowamixture.Atthesupermarket,hebuys
meatbythekilogramandfruitandvegetablesbythepound.He
onceboughta4-footby8-footsheetofplywoodthatwas19mm
thick.
Whiletheremaybemoreneedforconvertingtoandfrom
metricunitsinCanada,itisstillneededintheU.S.Mostofthe
worldusesthemetricsystem,andtheU.S.needstointeractwithit.
5.14conversion-funcs—ConvertingMetric
Units
Whilemostpeopleknowthecommonmetrictemperatures(0°Cis
32°F,thefreezingpointofwater,and20°Cisacomfortableroom
temperature),andtheyknowthatakilogramis2.2pounds,itmay
beastretchtoknowwhat32°CisinFahrenheit,orhowmany
poundsarein3.5kilograms.Shellscriptscandotheconversion
calculationsbetweenmetricandimperialmeasurementsystems.
HowItWorks
Unixsystemshavetraditionallyincludedautilitycalledunits.
Whileitisapowerfulprogramthatcanbeextendedbydefining
conversionsinanexternalfile,itisslow,andnoteasytouseina
script.Someversionscandoonlylinearconversions,which
excludesFahrenheittoCelsius,andsomeUnixsystemsdonot
installitbydefault.Therearesomescriptsforconversionsthatyou
couldusefrequently.Forthischapter,thereareevenmore.
Usage
CMD[UNITS]
_CMDUNITS
Likemostofthepreviousfunctionsinthischapter,thesecome
intwoflavors.Thefirstisforinteractiveuse,andwillpromptfor
inputifnoargumentisgiven.Thesecondexpectsanargument,and
storestheresultinavariable,whichisthenameofthefunction
(includingtheunderscore)inuppercase.
Thereare16commandsintheconversion-funcsscript,which
canbesourcedinashellinitializationfile(e.g.,$HOME/.profile—
seeChapter1formoreinformationonshellinitializationand
sourcingfiles)forinteractiveuse,oratthetopofascripttomake
themavailablethere:
f2c:DegreesFahrenheittoCelsius
f2k:DegreesFahrenheittokelvins
c2f:DegreesCelsiustoFahrenheit
k2f:KelvinstodegreesFahrenheit
m2km:Milestokilometers
km2m:Kilometerstomiles
y2m:Yardstometers
m2y:Meterstoyards
i2cm:Inchestocentimeters
i2mm:Inchestomillimeters
cm2i:Centimeterstoinches
mm2i:Millimeterstoinches
lb2kg:Poundstokilograms
kg2lb:Kilogramstopounds
oz2g:Ouncestograms
g2oz:Gramstoounces
Thesefunctionsarealsoavailablethroughtheconversionmenu
scriptthatfollowsthefunctions.
TheScript
Firstthemath-funcslibraryisloaded,whichinturnloadsthe
standard-funcslibrary.
.math-funcs
Sixteenconversionsmakeuptheconversion-funcsscript,each
ofwhichhastwofunctions:onetodothecalculation,andoneto
printtheresult.Whenusedinascript,ratherthaninteractively,the
resultneednotbeprinted,andthescriptwillgettheresultfromthe
variablesetineachfunction.
1.ConvertDegreesFahrenheittoCelsius
f2c()
{
units="degreesCelsius"
prompt="DegreesFahrenheit"arg"$@"
_f2c"$arg"
printf"%s%s\n""$_F2C""${units:+$units}"
}
_f2c()
{
eval"ècho$1|
awk'{
printf"_F2C=%"pr"f\n",($1-32)*
0.555556
}'pr=${precision:-.2}`"
}
2.ConvertDegreesFahrenheittoKelvins
f2k()
{
prompt="DegreesFahrenheit"arg"$@"
echo"$arg"|
awk'{
printf"%"pr"f\n",($1-32)*
0.555556+273.15
}'pr=${precision:-.2}
}
_f2k()
{
eval"ècho$1|
awk'{
printf"_F2K=%"pr"f\n",($1-32)*
0.555556+273.15
}
3.ConvertDegreesCelsiustoFahrenheit
c2f()
{
prompt="DegreesCelsius"arg"$@"
echo"$arg"|
awk'{
printf"_C2F=%"pr"f\n",$1*1.8+
32
}'pr=${precision:-.2}
}
_c2f()
{
eval"ècho"$1"|
awk'{
printf"_C2F=%"pr"f\n",$1*1.8+32
}
4.ConvertKelvinstoDegreesFahrenheit
k2f()
{
units="degreesFahrenheit"
prompt="Kelvins"arg"$@"
echo"$arg"|
awk'{
printf"%"pr"f%s\n",($1-273.15)
*1.8+32,units
}'pr=${precision:-.2}units="${units:+
$units}"
}
_k2f()
{
eval"ècho"$1"|
awk'{
printf"_K2F=%"pr"f\n",($1-
273.15)*1.8+32
}
5.ConvertMilestoKilometers
m2km()
{
units=kilometers
prompt=Milesarg"$@"
_m2km"$arg"
printf"%s%s\n""$_M2KM""${units:+$units}"
}
_m2km()
{
_fpmul"$1"1.609344
_M2KM=$_FPMUL
}
6.ConvertKilometerstoMiles
km2m()
{
units=miles
prompt=Kilometersarg"$@"
_fpmul"$arg".62135
printf"%s%s\n""$_FPMUL""${units:+$units}"
}
_km2m()
{
_fpmul"$1".62135
_KM2MILE=$_FPMUL
}
7.ConvertYardstoMeters
y2m()
{
units=meters
prompt=Yardsarg"$@"
_y2m"$arg"
printf"%s%s\n""$_Y2M""${units:+$units}"
}
_y2m()
{
_fpmul"$1"0.914402
_Y2M=$_FPMUL
}
8.ConvertMeterstoYards
m2y()
{
units=yards
prompt=Metersarg"$@"
_m2y"$arg"
printf"%s%s\n""$_M2Y""${units:+$units}"
}
_m2y()
{
_fpmul"$1"1.09361
_M2Y=$_FPMUL
}
9.ConvertInchestoCentimeters
i2cm()
{
units=centimeters
prompt=inchesarg"$@"
_i2cm"$arg"
printf"%s%s\n""$_I2CM""${units:+$units}"
}
_i2cm()
{
_fpmul"$1"2.54
_I2CM=$_FPMUL
}
10.ConvertInchestoMillimeters
i2mm()
{
units=millimeters
prompt=inchesarg"$@"
_i2mm"$arg"
printf"%s%s\n""$_I2MM""${units:+$units}"
}
_i2mm()
{
_fpmul"$1"25.4
_I2MM=$_FPMUL
}
11.ConvertCentimeterstoInches
cm2i()
{
units=inches
prompt=Millimetersarg"$@"
_cm2i"$arg"
printf"%s%s\n""$_MM2I""${units:+$units}"
}
_cm2i()
{
_fpmul"$1".3937
_CM2I=$_FPMUL
}
12.ConvertMillimeterstoInches
mm2i()
{
units=inches
prompt=Millimetersarg"$@"
_cm2i"$arg"
printf"%s%s\n""$_CM2I""${units:+$units}"
}
_mm2i()
{
_fpmul"$1".03937
_MM2I=$_FPMUL
}
13.ConvertPoundstoKilograms
lb2kg()
{
units=kilograms
prompt=Poundsarg"$@"
_lb2kg"$arg"
printf"%s%s\n""$_LB2KG""${units:+$units}"
}
_lb2kg()
{
_fpmul"$1".4545
_LB2KG=$_FPMUL
}
14.ConvertKilogramstoPounds
kg2lb()
{
units=pounds
prompt=Kilogramsarg"$@"
_kg2lb"$arg"
printf"%s%s\n""$_KG2LB""${units:+$units}"
}
_kg2lb()
{
_fpmul"$1"2.2
_KG2LB=$_FPMUL
}
15.ConvertOuncestoGrams
oz2g()
{
units=grams
prompt=Ouncesarg"$@"
_oz2g"$arg"
printf"%s%s\n""$_OZ2G""${units:+$units}"
}
_oz2g()
{
_fpmul"$1"28.35
_OZ2G=$_FPMUL
}
16.ConvertGramstoOunces
g2oz()
{
units=ounces
prompt=Gramsarg"$@"
_g2oz"$arg"
printf"%s%s\n""$_G2OZ""${units:+$units}"
}
_g2oz()
{
_fpmul"$1"0.0353
_G2OZ=$_FPMUL
}
5.15conversion—AMenuSystemforMetric
Conversion
Amenusystemwouldmakeusingtheconversionfunctionseasier,
atleastuntilonehasmemorizedallthefunctionnames.
HowItWorks
The16commandsaredividedinto4groups,withamenuforeach
group.Asinglekeystrokeinvokeseachmenuitem.Thenamesof
thefunctionsareincludedinthemenustohelpusersmemorize
them.
Usage
conversion
Thisscripttakesnooptionsandrequiresnoarguments.Atthe
mainmenu,pressinganumberfrom1to4willtakeyoutothe
menufortheselectedcategoryofconversion.Ineachmenu,one
keystrokewillbringupaprompttoenterthenumberyouwant
converted.Theanswerwillbeprinted,andthescriptwillwaitfor
anotherkeystrokebeforereturningtothemenu.
Inthissamplerun,theuserselectsweightconversions,then
ouncestograms,andenters2.Thescriptprintstheanswerand
promptsforakeypressbeforereturningtothemenu.Pressing0
takestheusertothepreviousmenu,andanother0exitsthescript.
Thecharactersinboldaretheuser’sinput,whichdonotactually
appearwhenthescriptisrun.
$conversion
=============================
Conversionfunctions
=============================
0.Exit
1.Lengthsanddistances
2.Weights
3.Temperature
=============================
Select[0-3]:2
===================================
Weights
===================================
0.Exit
1.lb2kg-poundstokilograms
2.kg2lb-kilogramstopounds
3.oz2g-ouncestograms
4.g2oz-gramstoounces
===================================
Select[0-4]:3
Ounces?2
56.7grams
<PRESSANYKEY>
===================================
Weights
===================================
0.Exit
===================================
Select[0-4]:0
=============================
Conversionfunctions
=============================
0.Exit
2.Weights
3.Temperature
=============================
Select[0-3]:0
$
TheScript
Fourfunctionsfromstandard-funcsareusedinthismenuprogram:
menu1:Createsasimplemenusystem.
get_key:Returnsakeypressedbytheuser.
press_any_key:Promptstheusertopressanykey.
cleanup:Calledfromatrap,cleanuprestoresanyterminal
settingsthathavebeenchanged.
Theconversionscripthasaseparatefunctionforthemainmenu
andforeachsubmenu.Morecategoriescanbeaddedeasily.
progname=${0##*/}
##ifthisisthedevelopmentversion
ofthescript(i.e.endswith"-sh"),
##usethedevelopmentversionofthe
functionlibraries
##(moreonthisappearsinthe
ScriptDevelopmentSystemchapter)
case$prognamein
*sh)shx=-sh;;
esac
trapcleanupEXIT
.conversion-funcs$shx
TheMainMenu
main_menu()
{
menu="
=============================
Conversionfunctions
=============================
0.Exit
2.Weights
3.Temperature
=============================
Select[0-3]:"
menu1distancesweightstemperatures
}
TheLengthsandDistancesMenu
distances()
{
menu="
==================================
Lengthsanddistances
==================================
0.Exit
1.m2km-milestokilometers
2.km2m-kilometerstomiles
3.y2m-yardstometers
4.m2y-meterstoyards
5.i2cm-inchestocentimeters
6.cm2i-centimeterstoinches
7.i2cm-inchestomillimeters
8.cm2i-millimeterstoinches
==================================
Select[0-8]:"
pause_after=1menu1m2kmkm2my2mm2yi2cmcm2ii2mm
mm2i
}
TheWeightsMenu
weights()
{
menu="
===================================
Weights
===================================
0.Exit
===================================
Select[0-4]:"
pause_after=1menu1lb2kgkg2lboz2gg2oz
}
TheTemperatureConversionMenu
temperatures()
{
menu="
=====================================
Temperatures
=====================================
0.Exit
1.f2c-FahrenheittoCelsius
2.c2f-CelsiustoFahrenheit
3.f2k-Fahrenheittokelvin
4.k2f-kelvinstodegreesFahrenheit
=====================================
Select[0-4]:"
pause_after=1menu1f2cc2ff2kk2f
}
precision=.2
main_menu
printf"\n"
Summary
Thischapterhasbarelyscratchedthesurfaceofthenumerical
manipulationspossiblewithshellscripts.Therearemoreinother
chapters,andnodoubtyoucanthinkofmanymore,too.
Mostofthesescriptsareverysimple;aprogramdoesn’thaveto
becomplicatedtobeuseful.Youcancreateanewmetric
conversionscriptifyouknowtheformula,andanythingcanbe
foundontheInternet.(Youdon’tthinkwerememberedthemall,do
you?)
Chapter6:LooseNamesSink
Scripts:BringingSanityto
Filenames
(1) Ontario,Canada
TheNamingofFilesisaseriousmatter,
Thisisn’tjustoneofyourUsenetflames;
Youmaythinkatfirstitdoesn’tmuchmatter
ButItellyou,afileshouldhaveaSENSIBLENAME.
(withapologiestoT.S.Eliot)
Whatisasensiblename,andwhatcanbedoneabout
pathologicalfilenames?Inthischapter,wedescribealternativesfor
creatingacceptablefilenamesandprovidescriptsfordealingwith
badnames.Wealsoprovideshellreplacementfunctionsforthe
basenameanddirnamecommands.
What’sinaName?
AUnixfilenamemaycontainanycharacterexceptNULandthe
forwardslash(/).ANUL(acharacterinwhichallbitsare0)cannot
bepartofafilenamebecausetheshellusesitinternallytosignalthe
endofastringofcharacters.Theslashisreservedforseparating
directoriesinthepathtoafile.Apartfromthosetwo,anycharacter
maybeusedinafilename.Thisisnottosaythatallofthe
permissiblecharactersshouldbeusedinfilenames.
Intheearly(andnot-so-early)daysofUnix,mostfile
manipulationwasdoneatthecommandlineorinashellscript.
Syntactically,afilenameisawordortoken,andthat’swhatthe
shellisdesignedtohandle.Puttingspacesinafilenamemakesno
senseinthatcontext.Thesameistrueofanumberofother
charactersthatcauseproblems:apostrophesandasterisks,brackets
andbackslashes,tonameafew.Theworstisthenewline(NL).
Characterssuchasspaces,asterisks,andnewlinescanbreak
many(especiallyolder)scripts.Whitespacecharacters,spaceand
tab,canusuallybeaccommodatedfairlyeasilybyquotingall
variablescontainingfilenames.Theydocauseproblemsinscripts
thatpipealistoffiles.
We’llcdtoanemptydirectoryandcreatesomefileswithspaces
intheirnames(thisusesbraceexpansionfrombash;it’snot
availableinallPOSIXshells):
$touch{qwe,rty,uio}\{asd,fgh,jkl}
Thenwe’lllistthem,onetoaline:
$ls-1*
qweasd
qwefgh
qwejkl
rtyasd
rtyfgh
rtyjkl
uioasd
uiofgh
uiojkl
Notethattheuseofawildcardpreservestheintegrityofthe
filenames.Thisintegrityislostwhenthefilenamesaretheoutputof
anothercommand;then,eachwordistreatedseparately:
$printf"%s\n"*|wc-l
9
$printf"%s\n"`ls*`|wc-l
18
$ls|xargsprintf"%s%s%s\n"
qweasdqwe
fghqwejkl
rtyasdrty
fghrtyjkl
uioasduio
fghuiojkl
Frequently,filesarepipedtoanothercommandthatreadsthem
linebyline.Ifnewlinesareinthefilenames,thismethodobviously
fails.
POSIXPortableFilenames
ThePOSIXstandard(strictlyspeaking,IEEEStd1003.1-2001)
definesaPortableFilenameCharacterSet1thatisguaranteedtobe
portableacrossPOSIXsystems.ModernUnixsystemsarelargely
POSIXcompliant,andthischaracterset,whichcomprisesthe52
upper-andlowercaseletters,the10digits,andtheperiod,hyphen,
andunderscore,issafeonallsuchsystems(aswellasmostolder
ones).Portablefilenamesmaynotbeginwithahyphen.2
Aportablefilenamewillnotrunafoulofanyshellscripts(well,
thereareways,buttheyneedn’tconcernushere).
OKFilenames
Foryourownuse,youcangetawaywithalargercharacterset.I
definethissetbythecharactersthatarenotallowed.Theseinclude
thewhitespacecharacters(NL,CR,tab,andspace),asterisk,question
mark,parentheses,squarebrackets,anglebrackets,braces,
apostrophe,doublequotes,octothorpe(#),andpipe(|).Youmay
wishtoaddmorecharacters.
FunctioningFilenames:Thefilename-funcs
Library
Thefilename-funcslibrarysourcesthestandard-varsandstringfuncsfiles(fromChapters1and3,respectively),setsvariablesthat
definetheportableandOKcharactersets,anddefinesanumberof
functionsformanipulatingfilenames.
##sourcethevariableandfunctionlibraries
.standard-vars$shx
.string-funcs$shx
##definecharactersforportableandOK
charactersets
##pfname_charsisalistofacceptable
characters
pfname_chars="a-zA-Z._0-9-"
##xfname_charsisalistofunacceptable
characters
xfname_chars="][|#?>
<*${BKS}\$'\"${SPC}${NL}${TAB}${CR}${BS}${DEL}()"
Mostfunctionsinthisfilefollowtheconventionusedearlierof
pairingthefunctions;onesetsavariable,andtheothercallsthefirst
one,thenprintsthevariable.
Thefirsttwofunctions,basenameanddirname,areshell
implementationsofthestandardcommands.Theyconformtothe
POSIXspecification,3butarefasterbecausetheydonotstartanew
process.Whilemostofthefunctionscanbeusedontheirown,the
lasttwo,fix_filenameandfix_pwd,areintendedtobecalledbythe
fixfnamecommandthatfollows.
6.1basename—ExtracttheLastElementofa
Pathname
Thebasenamecommandextractsthelastelement,usuallyafilename,
fromapathname.Optionally,italsoremovesasuffixfromthat
element.
HowItWorks
InaPOSIXshell,thebasenamecommandisrarelyneeded.
Parameterexpansioncanextractthefilenamefromapathname:
$pathname=/home/jayant/work/book/chapter05.doc
$filename=${pathname##*/}
$printf"%s\n""$filename"
chapter05.doc
Itcanalsoremoveasuffix:
$printf"%s\n""${filename%.doc}"
chapter05
ThisbasenamefunctionaddsfullPOSIXcompliancetothe
parameterexpansion;itstripstrailingslashesbeforeextractingthe
basename,anddoesnotremoveasuffixifnothingwouldbeleft.It
isafullreplacementfortheexternalcommand,andscriptsthatuse
basenamedonotneedtobemodified.
Usage
_basenamePATHNAME[SUFFIX]##result
isstoredin$_BASENAME
basenamePATHNAME[SUFFIX]##result
isprinted
Ifavariablecontainsthepathtoafile(e.g.,
script=/home/jayant/bin/wf-sh),thefilenamealonecanbeprinted
withbasename:
$basename$script
wf-sh
Toprintthefilenamewithoutthe-shsuffix,usethis:
$basename$script-sh
wf
Ifthepathgiventobasenameendsinoneormoreslashes,then
theslashesareremoved,andthelastpartoftheremainingpathis
returned:
$basename/usr/local/bin/
bin
TheScript
Thestrip_trailing_slashesfunctionisusedbybothbasenameand
dirname.
strip_trailing_slashes()
{
_STRIP=$1
while:
do
case$_STRIPin
##Ifthelastcharacerisaslash,remove
it
*/)_STRIP=${_STRIP%/};;
##otherwiseexittheloop(andfunction)
*)break;;
esac
done
}
_basename()##extractthelast
elementfromapathname
{
fn_path=$1
fn_suffix=$2
case$fn_pathin
##Thespecsays:"Ifstringisanullstring,
itis
##unspecifiedwhethertheresultingstringis
'.'ora
##nullstring."Thisimplementationreturnsa
nullstring
"")_BASENAME=;return;;
*)
strip_trailing_slashes"$fn_path"
case$_STRIPin
"")fn_path="/";;
*)fn_path=${_STRIP##*/};;
esac
;;
esac
##Ifasuffixwasgiven,anditisthefinal
portionofthefilename(butnot
##thewholename),thenremoveit.
case$fn_pathin
$fn_suffix|"/")_BASENAME="$fn_path";;
*)_BASENAME=${fn_path%$fn_suffix};;
esac
}
basename()
{
_basename"$@"&&
printf"%s\n""$_BASENAME"
}
6.2dirname—ReturnAllbuttheLastElement
ofaPathname
Returnallofapathnameexceptthepartthatwouldbereturnedby
basename.Inotherwords,printthedirectorythatcontainsthelast
elementofthepathname.
HowItWorks
Thedirnamefunctionisafull,POSIX-compliantreplacementfor
theexternaldirnamecommand.
Usage
_dirnamePATHNAME##resultisstored
in$_DIRNAME
dirnamePATHNAME##resultisprinted
Likebasename,dirnamedoesnothonorthePOSIXstandardof
interpretingapathnameendingwithaslashasifitendedwithslashdot(/.).4Alltrailingslashesareremovedbeforethelastelementis
removed:
$dirname/home/chris/Chess/
/home/chris
ThePOSIXdefinitionofdirnameallowstheimplementationto
definewhetherapathnameconsistingof//returnsoneortwo
slashes.Thisimplementationreturnsasingleslash:
$dirname//
/
TheScript
_dirname()
{
_DIRNAME=$1
strip_trailing_slashes
case$_DIRNAMEin
"")_DIRNAME='/';return;;
*/*)_DIRNAME="${_DIRNAME%/*}";;
*)_DIRNAME='.';return;;
esac
strip_trailing_slashes
_DIRNAME=${_DIRNAME:-/}
}
dirname()
{
_dirname"$@"&&
printf"%s\n""$_DIRNAME"
}
6.3is_pfname—CheckforPOSIXPortable
Filename
Whethercheckingexistingfilesorvalidatingauser’sentry,a
methodisneededtodeterminewhetherafilenameconformstothe
POSIXportablefilenamestandarddescribedatthebeginningofthis
chapter.
HowItWorks
Useacasestatementtocheckwhetherthefilenamecontains
anythingotherthancharactersfromtheportablecharacterset.
Successfulcompletion(returncode0)indicatesthatthename
containsonlyacceptablecharactersandthatthefilenamedoesnot
beginwithahyphen.Anemptyfilenamewillalsofailthetest.
Usage
is_
pfname
"FILENAME"
Thereturncodecanbetestedexplicitly,orthedecisioncanbe
madedirectlyonsuccessorfailureofthecommand:
##Testthereturncode
is_pfname"$FILENAME"
if[$?-eq0]
then
printf"%sisacceptable\n""$FILENAME"
else
printf"%sneedstobefixed\n""$FILENAME"
fi
##Testforsuccessorfailure
ifis_pfname"$FILENAME"
then
printf"%sisacceptable\n""$FILENAME"
else
printf"%sneedstobefixed\n""$FILENAME"
fi
NamesthatconformtothePOSIXportablefilenamestandard
includeChapter_1.html,half-way.txt,and4711-ch06.doc.
Unacceptablefilenamesincludewilliam&mary(theampersandisnot
allowed)and-lewis_and_clark(aleadinghyphenisforbidden).
TheScript
is_pfname()
{
pfname_test="[!$pfname_chars]"##Patterntousein
thecasestatement
case$1in
##Testforemptyfilename,leadinghyphen,or
deprecatedcharacters
""|-*|*$pfname_test*)return1;;
esac
}
6.4is_OKfname—CheckWhetheraFilenameIs
Acceptable
CheckingwhetherafilenameconformstothelooserOKstandard
describedatthebeginningofthechapterissimilartocheckingfora
POSIXportablefilename.Itmustnotacceptanemptyname,a
namebeginningwithahyphen,oronecontaininganycharacterin
theproscribedlists.
HowItWorks
Acasestatement,similartotheoneinis_pfname,checksforall
threeconditions.
Usage
is_
OKfname
"FILENAME"
Thisfunctioncanbeusedinexactlythesamewayasis_pfname.
Bothfunctionscanalsobeusedinalooptolistallnonconforming
files:
forfilein*
do
is_OKfname||printf"::%s\n"
"$file"
done
TheScript
is_OKfname()
{
xfname_test="[$xfname_chars]"##Patterntousein
thecasestatement
case$1in
##Testforemptyfilename,leadinghyphen,or
deprecatedcharacters
""|-*|*$xfname_test*)return1;;
esac
}
6.5pfname—ConvertNonportableCharactersin
Filename
Theprevioustwofunctionscheckedwhetherafilenameconformed
toanacceptablecharacterset.Thenextstepistoconvertafilename
toaportablefilename(or,asinthescriptfollowingthisone,anOK
filename).
HowItWorks
Thepfnamefunctionuses_gsubfromthestring-funcslibrary
(describedinChapter3)tochangeallnonportablecharacterstoan
acceptablecharacter:bydefault,anunderscore.Anothercharacter
(orevenastringofcharacters)maybeusedbyassigningitto
rpl_char.Aleadinghyphenisconvertedtoanunderscore.
Usage
_pfname"FILENAME"##resultis
storedin$_PFNAME
pfname"FILENAME"##resultis
printed
Ifthefilenameisalreadyportable,pfnamewillnotchangeit:
$FILE=qwerty.uiop
$pfname"$FILE"
qwerty.uiop
Ontheotherhand,aninvalidcharacterwillbeconvertedtoan
underscore:
$pfname"qwerty=uiop"
qwe_rty_uiop
Thisshortscriptusespfnametofixafilenameifitisnot
portable:
is_pfname"$FILE"||{
_pfname"$FILE"
mv"$FILE""$_PFNAME"
}
Acharacterotherthananunderscorecanbeusedasthe
replacementcharacter:
$FILE=qwe=+rty
$rpl_char=X
$pfname"$FILE"
qweXXrty
Toallowmorecharacters,addthemtothepfname_chars
variable:
$pfnameqwe+rty=asd
qwe_rty_asd
$pfname_chars=${pfname_chars}+=
$pfnameqwe+rty=a*sd
qwe+rty=a_sd
Sincetheplusandequalsignswereaddedtothelistof
acceptablecharacters,onlytheasteriskwaschanged.
TheScript
_pfname()
{
pfname_pat="[!$pfname_chars]"##Patternforcase
statement
_gsub"$1""$pfname_pat""${rpl_char:-_}"##Change
unacceptablecharacters
case$_GSUBin
-*)_PFNAME=_${_GSUB#?};;##Convertleading
hyphentounderscore
*)_PFNAME=$_GSUB;;
esac
}
pfname()
{
_pfname"$@"&&
printf"%s\n""$_PFNAME"
}
6.6OKfname—MakeaFilenameAcceptable
Removeproblematiccharactersfromafilenamebyusingamore
liberalsetofcharactersthanthePOSIXportablecharacterfilename
set.
HowItWorks
Unlikepfname,whichchangeseverythingnotincludedinasetof
acceptablecharacters,OKfnameconvertsunacceptablecharactersto
underscore(oranothercharacterorstring).
Usage
_OKfname"FILENAME"##resultis
storedin$_OKFNAME
OKfname
"FILENAME"##resultisprinted
UsageofOKfnameisthesameaspfname,exceptthatthevariable
xfname_charscontainscharactersthatshouldnotbeallowed:
$fname=qwe+rty=asd
$OKfname"$fname"
qwe+rty=asd
Byaddingtheequalsigntothexfname_charsvariable,ittoowill
beconvertedtoanunderscore.
$xfname_chars=${xfname_chars}=
OKfname"$fname"
qwe+rty_asd
TheScript
_OKfname()
{
xfname_pat="[$xfname_chars]"
_gsub"$1""$xfname_pat""${rpl_char:-_}"
case$_GSUBin
-*)_OKFNAME=_${_GSUB#?};;
*)_OKFNAME=$_GSUB;;
esac
}
OKfname()
{
_OKfname"$@"&&
printf"%s\n""$_OKFNAME"
}
6.7is_whitespc—DoestheFilenameContain
WhitespaceCharacters?
Whitespacecharactersarethemostcommonprobleminfilenames,
soitisusefultohaveafunctiontodeterminewhetherafilename
containsaspace,atab,anewline,oracarriagereturn.
HowItWorks
Usecasepatternmatchingtocheckforwhitespacecharacters.
Usage
is_
whitespace
"FILENAME"
Whitespaceisbad,sois_whitespcfailsifanywhitespaceisin
thefilename.
TheScript
is_whitespc()
{
case$1in
*[$SPC$TAB$NL$CR]*)return1;;
esac
}
6.8whitespc—FixFilenamesContaining
WhitespaceCharacters
Whetherit’sbecausewemightknowthattheonlyproblem
charactersarespaces,orbecausetheothernonconforming
charactersarenotgoingtocauseanyproblems,sometimesallyou
mightwanttodoistoconvertwhitespacecharacterstounderscores.
HowItWorks
The_gsubfunctionfromthestring-funcslibraryinChapter3
comestotherescuetochangethosepeskycritterstoPOSIX-abiding
filenames.
Usage
_whitespc"FILENAME"##resultisin
$_WHITESPC
whitespc
Bydefault,spacesarechangedtounderscores:
$whitespc"ErictheRed"
Eric_the_Red
Touseadifferentreplacementcharacter,settherpl_char
variabletothatcharacterorstring:
$rpl_char=-whitespc"definitelynot
theopera"
definitely-not-the-opera
TheScript
_whitespc()##convertwhitespaceto
underscores
{
_gsub"$1""[${TAB}${NL}${CR}]""${rpl_char:-_}"
case$_GSUBin
-*)_WHITESPC=_${_GSUB#?};;
*)_WHITESPC=$_GSUB;;
esac
}
whitespc()
{
_whitespc"$@"
printf"%s\n""$_WHITESPC"
}
6.9is_dir—IsThisaDirectoryISeeBeforeMe?
Beforecallingacommandthattakesoneormoredirectoriesas
arguments,youwanttocheckwhether*/expandstoatleastone
directory.Youcannotusetest-d,becauseitonlyacceptsasingle“
youcancallwith*/asanargument,andhaveittellmewhetherthe
argumenthasexpandedsuccessfully.
HowItWorks
The-doperandtotest(usuallyusedas[)expectsonlyone
argument,sotest-d*/willfailifitexpandstomorethanone
directory.Byputtingthetestinafunction(itcouldalsobean
externalcommand),theshellexpandstheargument,andthe
functionseesalistofdirectories(ifthereareany).Thescriptonly
needstocheckthefirstargument.
Usage
is_
dir
PATTERN
ThefunctionwillfailifthefirstargumentwhenPATTERN
expandsisnotadirectory:
$is_dir*/&&some_command"$@"||
echo"Nodirectory">&2
Italsoworksifgivenasingleargument:
$is_dir/home/chris/grid.ps&&echo
YES||echoNO
TheScript
is_dir()##isthefirstargumenta
directory?
{
test-d"$1"
}
Notes
Thefunctionwasdesignedforusewith*/or/path/to/dir/*/as
thecallingpattern.Itdoesnotworkwithapatternthatmayexpand
toamixtureoffilesanddirectories,unlessadirectoryhappenstobe
first.Afunctiontoindicatewhetheranyoftheargumentsisa
directorycouldbewrittenlikethis:
any_dirs()
{
forx
do
test-d"$x"&&return
done
return1
}
6.10nodoublechar—RemoveDuplicate
CharactersfromaString
Afterremovingpathologicalcharactersfromafilename,oftenthere
willbetwoormoreconsecutiveunderscores.Thiscommandis
neededtoremoveduplicateunderscoresfromafilenameorother
string.
HowItWorks
Theubiquitous_gsubfunctionconvertstwocharacterstoone.
Usage
_nodoublechar"FILENAME"[CHAR]##
resultisin$_NODOUBLECHAR
nodoublechar
"FILENAME"[CHAR]##resultis
printed
Afterconvertingafilenamesuchastea&coffeewithpfnameor
OKfname,threeconsecutiveunderscoresremain.Thisfunction
reducesthemtoasingleunderscore:
$nodoublechar"teacoffee"
tea_coffee
TheScript
_nodoublechar()##removeduplicate
characters
{
_NODOUBLECHAR=$1##Thestringtomodify
_char=${2:-_}##Thecharacter(orstring)to
operateon
while:
do
case$_NODOUBLECHARin
##Ifthereisadoubleinstanceofthe
string,
##reducetoasingleinstance
*${_char}${_char}*)_gsub"$_NODOUBLECHAR"
"${_char}${_char}""${_char}"
_NODOUBLECHAR=$_GSUB
;;
*)break;;##Allclear;exittheloopand
function
esac
done
}
nodoublechar()
{
_nodoublechar"$@"&&
printf"%s\n""$_NODOUBLECHAR"
}
6.11new_filename—ChangeFilenameto
DesiredCharacterSet
Thisscriptwillconvertafilenameusingoneofthefourdefined
functions:
pfname:ConvertallcharactersnotinthePOSIXportable
filenamecharactersettounderscores.
OKfname:Convertallunacceptablecharacterstounderscores.
whitespc:Convertallwhitespacecharacterstounderscores.
nodoublechar:Reducemultipleunderscorestoasingle
underscore.
HowItWorks
Settheff_cmdvariabletothedesiredconversionfunctionandcall
new_filenamewiththeoldfilenameastheargument.Ifff_cmdisnot
set,itdefaultsto_pfname.Anothercharacter(orstring)canbeused
inplaceofanunderscorebyredefiningrpl_char.
Usage
_new_filename"FILENAME"##resultin
$_NEW_FILENAME
new_
filename
Thisfunctionisintendedtobecalledbythefunctionfix_pwd,
describedinthefollowingsection;seethatfunctionforitsuse.
Normally,theconversionfunctionwouldbecalleddirectlybya
script;thisisamechanismforselectingandcallingthecorrect
function.
TheScript
_new_filename()
{
${ff_cmd:=_pfname}"$1"
case$ff_cmdin
_pfname)_NEW_FILENAME=$_PFNAME;;
_OKfname)_NEW_FILENAME=$_OKFNAME;;
_whitespc)_NEW_FILENAME=$_WHITESPC;;
_nodoublechar)_NEW_FILENAME=$_NODOUBLECHAR;;
esac
[$nodoublechar-eq1]&&{
_nodoublechar"$_NEW_FILENAME"_
_NEW_FILENAME=$_NODOUBLECHAR
}
}
new_filename()
{
_new_filename"$@"
printf"%s\n""$_NEW_FILENAME"
}
6.12fix_pwd—FixAlltheFilenamesinthe
CurrentDirectory
Checkeachfileinthecurrentdirectoryandconvertanythatdonot
fittheprescribedcharacterset.
HowItWorks
Severalvariablescontrolthebehavioroffix_pwd:
verbose:Ifsetto1orgreater,printprogressinformation.
pattern:Changeonlythosefileswhosenamesmatchthe
patterncontainedinthevariable;thedefaultisallfilesif
patternisempty.
f_only:Onlymodifynamesoffiles.
d_only:Onlymodifynamesofdirectories.
ff_cmd:Usethefunctioncontainedinthevariabletomodify
thefilename(seefix_filename).
Usage
fix_
pwd
Thefunctioniscontrolledentirelybythevariablesjustlisted
andtakesnoarguments.
TheScript
fix_pwd()
{
##Printnameofdirectoryinverbosemode
[${verbose:=0}-ge1]&&printf"::%s::\n"
"${PWD%/}/">&2
##Loopthroughallfilesthatmatchthepattern
forfilein${pattern:-*}
do
[-e"$file"]||continue##shouldbe
unnecessary
##findfiletype
[-f"$file"]&&f_type=f
[-d"$file"]&&f_type=d
##onlyoperateonregularfilesanddirectories
case$f_type$d_only$f_onlyin
f0?|d?0);;
*)continue;;
esac
##checkwhetherfilenameisOK,andmodifyif
not
is${ff_cmd:=_pfname}"$file"||{
_new_filename"$file"
new_base=$_NEW_FILENAME
n=1
##if$_NEW_FILENAMEexists,appendanumber
while[-e"$_NEW_FILENAME"]
do
_NEW_FILENAME=$new_base.$n
[$verbose-ge1]&&printf"%s\r"
"$_NEW_FILENAME">&2
n=$(($n+1))
done
[$verbose-ge1]&&
printf"%s->%s""$file"
"$_NEW_FILENAME">&2
mv"./$file""./$_NEW_FILENAME"&&[
$verbose-ge1]&&
printf"OK\n">&2||printf
"failed\n">&2
}
done
}
6.13fixfname—ConvertFilenamestoSensible
Names
Filenamescontainingspacesortabsareannoying.Filenames
beginningwithahyphenmaybeconfusedwithcommandoptions.
Filenamescontaininganapostrophemaybeconsideredincomplete.
Filenamescontaininganewlineareanabomination.Allofthese,
andmore,canbreakscripts.
Unlessthere’sagoodreasonnotto(suchasanapplicationthat
expectstofindbadlynamedfiles),thesefilenamesshouldbefixed.
Attheveryleast,allwhitespacecharactersshouldberemoved,
perhapsconvertedtounderscores.Atthemost,allfilenamesshould
conformtothePOSIXportablefilenamestandard.Areasonable
compromiseistogetridofthemosttrouble-somecharacters,the
OKcharacterset.
HowItWorks
Hereisascriptthatwillconvertallthefilenamesinadirectoryto
yourchoiceofPOSIX,OK,orjustwhitespace-free.Theunsavory
charactersarereplacedbyunderscores(oranothercharacterofyour
choice).Youcanchoosetosqueezemultipleunderscorestoasingle
character.
Youcanconvertthecurrentdirectory,oryoucandescenda
directorytree(excludingsymboliclinks).Youcanconvertonly
directorynames,oronlyfilenames.Youcanlimittheconversionto
namesthatmatchapattern.
Whenaconvertedfilenamematchesanexistingfilename,a
numberisaddedtotheendofthename.Thisnumberis
incrementedasnecessarytomakethefilenameunique.
Usage
fixfname
[OPTIONS][DIRECTORY...]
Command-lineoptionscontrolthebehavioroffixfname,
includingthetypeofconversion(bydefault,fixfnameusesthe
POSIXstandard):
-cC:UseCtoreplaceundesirablecharactersinsteadofan
underscore.
-k:UsethemoreliberalOKcharactersetinsteadofPOSIX.
-s:Justconvertthewhitespacecharacters:space,tab,newline,
andcarriagereturn.
-u:Replacedouble(ortriple,ormore)underscoreswitha
singleunderscore.Donotperformanyotherconversion.
-d:Convertdirectorynames,butleavefilenamesalone.This
doesnotaffectrecursion.
-f:Changefilenames;directoryassistanceisnotrequired.
-pPAT:TransformonlythosenamesthatmatchPAT,using
pathnameexpansion.Avoidusingproblematiccharacters
(spaces,etc.)inPAT.
-U:Convertmultipleunderscores,aswith-u,butcombined
withanotherconversion.
-v:Verbosemode;print(tostandarderror)thenamesof
directoriesthatareentered,andtheoldandnewfilenames
whenaconversionismade.
-R:Descenddirectoriesrecursively(butdonotdescend
throughsymboliclinks).
-V:Printtheversionnumberofthescriptandexit.
Withfixfname,youcanoperateonanythingfromasinglefileto
anentiredirectoryhierarchy.Hereareafewexamples.
Toconvertallspacesinthe.jpgfilenamesinthecurrent
directorytounderscores,usethis:
fixfname-sp"*.jpg"
Toconvertall.oggfilesinthecurrentdirectorytreetoportable
filenameswithallmultipleunderscoresreducedtoasingleone,use
this:
fixfname-Rup"*.ogg"
[email protected]
andallitssubdirectories,usethis:
fixfname-Rvp'[email protected]'
TheScript
progname=${0##*/}##thiscanusually
replace`basename$0`
##Whileunderdevelopment,my
scriptshavenamesendingwith"-sh"
##Thesescriptscallthedevelopment
versionsoffunctionlibraries
##whosenamesalsoendin-sh.(The
scriptdevelopmentsystem,whichcopies
##thescripttothebindirectory
andremovesthesuffix,ispresentedin
##Chapter20.)
case$prognamein
*-sh)shx=-sh;;
*)shx=;;
esac
##filename-funcs[-sh]containsall
theprecedingfunctions
.filename-funcs$shx
##Setdefaultsverbose=0
recurse=0##Donotdescend
intodirectories
ff_cmd=_pfname##UsePOSIX
filenames
rpl_char=_##Replacebad
characterswithunderscore
pattern=*##Allfilesin
currentdirectory
nodoublechar=0##Donotsqueeze
doubleunderscores
f_only=0##Donotmodify
onlydirectories
d_only=0##Donotmodify
onlyfiles
R_opt=##Clearoptionsfor
recursiveoperation
whilegetoptsvRUVfdksuc:p:var
do
case$varin
c)rpl_char=$OPTARG##replacementforunderscore
R_opt="$R_opt-c'$rpl_char'";;
k)ff_cmd=_OKfname##useOKfilenamesrather
thanportableones
R_opt="$R_opt-k";;
s)ff_cmd=_whitespc##justconvertwhitespaceto
underscores
R_opt="$R_opt-s";;
u)ff_cmd=_nodoublechar##convertdouble
underscorestosingle
R_opt="$R_opt-u";;
d)d_only=1##onlyprocessdirectories
R_opt="$R_opt-d";;
f)f_only=2##onlyprocessfiles
R_opt="$R_opt-f";;
op)pattern=$OPTARG##only
changenamesmatching$pattern
R_opt="$R_opt-p'$pattern'";;
U)nodoublechar=1##convert__[...]to_(with
otherfixes)
R_opt="$R_opt-U"
;;
v)verbose=$(($verbose+1))##printprogress
reports
R_opt="$R_opt-v"
;;
R)recurse=1;;##descenddirectories
recursively
V)printf"$progname:%s\n"$version;exit;;
esac
done
shift$(($OPTIND-1))
##Ifthereisadirectoryonthe
commandline.....
if[-d"$1"]
then
fordir##...enteritandanyotherdirectoriesin
turn
do
([-L"${dir%/}"]&&continue##donotdescend
symlinkeddirectories
cd"$dir"||continue##Skiptonextifdirnot
cd-able
fix_pwd##Doit
[$recurse-eq1]&&is_dir./*/&&eval"\"$0\"
-R$R_opt./*/"
)
done
else
fix_pwd
[$recurse-eq1]&&is_dir./*/&&eval"\"$0\"-R
$R_opt./*/"
fi
Summary
Ihopeyouhavenodoubtaboutmyfeelingsregardingbadlyformed
filenames.Ifshellscriptswereneverused,itwouldn’tmatter.But
despitethepopularityoflanguagessuchasPerlandPython,itis
exceedinglyunlikelythatshellscriptswilleverdisappearentirely.
Theyareanaturalcomplementtothecommandline.
Youmayhaverealizedthat,despitemyrantingover
@!$#%^*&filenames(I’verunoutofpolitethingstocallthem),
Youhaveachapterfullofscriptsthatcanprocessanyfile-names
youcancomeupwith.Itispossibletodoit,butalotofscripting
techniquesareruledoutbecauseofthem.Ifyouneedtowriteshell
scriptsthatworkonspaced-outfilenames,remembertwothings:
1. Populatevariableswiththeresultsoffilenameexpansion
(wildcards),nottheoutputofcommandssuchascatorls.
2. Enclosethevariablesinquoteswhenyouusethem:"$filename"
not$filename.
Footnotes
1 http://www.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap03.html#tag_
2 http://www.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap04.html#tag_
3 http://www.opengroup.org/onlinepubs/009695399/utilities/basename.htmland
http://www.opengroup.org/onlinepubs/009695399/utilities/dirname.html
4 “Apathnamethatcontainsatleastonenon-slashcharacterandthatendswithoneor
moretrailingslashesshallberesolvedasifasingledotcharacter(‘.’)wereappendedto
thepathname.”
http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_
Chapter7:TreadingaRighteous
PATH
(1) Ontario,Canada
ThePATHvariableisacolon-separatedlistofdirectoriesthatyour
shellwillsearchforacommand.Itusuallyincludes/bin,/usr/bin,
and/usr/local/bin.Ifyouareroot,itwillalsoinclude/sbinand
/usr/sbin.Theremaybeotherdirectories,andtheremayevenbea
dot,whichputsyourcurrentdirectoryinthePATH.Forexample,a
typicaluser’s$PATHlookslikethis:
/usr/bin:/bin:/usr/local/bin:/usr/games:/usr/X11R6/bin:/home/c
Wheneveryouenteracommandwithoutafullpath(e.g.,ls
insteadof/bin/ls),theshellsearchesthedirectorieslistedinyour
PATHvariableforthecommand.Ifallthedirectoriesyouneedare
notincluded,commandswillnotbefound.Ifthecurrentdirectoryis
included,especiallynearthefront,therearesecurityrisks.If
spuriousandduplicatedirectoriesareincluded,thesearchfor
commandsmaybeinfinitesimallyslower;it’snotreallyaproblem,
butwhynotkeepthingsneatandtidy?
Inthischapter,youhavealibraryoffunctionsformanipulating
yourPATHvariable:toaddanddeletedirectories,tocheckthatall
entriesarevalid,andtodisplayyourPATHinamorereadableform.
Ifyousourcepath-funcs,andruncheckpathinyour.profile,you
canviewyourPATHwithpathandmanipulateitwithaddpathand
rmpath,avoidinganypotentialproblems.
Thepath-funcsLibrary
YoucanusethefunctionsinthislibrarytokeepyourPATHvariable
ingoodshapebyaddingandremovingdirectories,ensuringthatall
itselementsare,infact,directories,andthatthecurrentdirectoryis
notamongthem.
7.1path—DisplayaUser’sPATH
For15years,thecomputerChrisusedwasanAmiga;ithadapath
command.Whencalledwithoutanyarguments,itwouldprintyour
commandpathonedirectoryperline—aformatthatismorelegible
thanasinglecolon-separatedline,whichmaybreakadirectoryin
twoifthePATHislongerthanascreenwidth.Iftherewereoneor
morearguments,itwouldaddthoseargumentstothecommand
path.Hemissedthatcommand,sohewroteaUnixversion.
HowItWorks
SincePATHisacolon-separatedlist,itcanbesplitbyassigninga
colontotheinternalfieldseparator,IFS.Asingleprintfstatement
printseachdirectoryonaseparateline.Ifoneormoredirectoriesis
givenasanargument,thePATHisnotprintedandtheaddpath
function(introducedlaterinthischapter)isusedtoappendthemto
thePATHvariable.
Usage
path[DIR...]
MyPATHcontainstwosubdirectoriesofmy$HOMEdirectory,as
wellasthesbindirectoriesusuallyreservedfortherootuser:
$path
/bin
/usr/bin
/usr/X11R6/bin
/usr/local/bin
/home/chris/bin
/home/chris/scripts
/usr/local/kde/bin
/sbin
/usr/sbin
/usr/local/sbin
TheScript
path()
{
if[-n"$*"];then
addpath"$@"||return
else
(##UseasubshellsoIFSisnotchangedin
themainshell
IFS=:
printf"%s\n"${PATH}
)
fi
}
7.2unslash—RemoveExtraneousSlashes
Thefunctionsinthischapterthatalterthe$PATH(checkpath,
addpath,andrmpath)comparedirectorynames,sotheyneedtohave
astandardizedformat.Thismeansremovingtrailingslashesand
collapsingdoubleslashestosingleslashes.
HowItWorks
Trailingslashesareeasilyremovedwithparameterexpansion.
Doubleslashesarechangedtosingleslasheswith_gsub,from
Chapter3’sstring-funcslibrary.Ifthelibraryhasnotbeenloaded,
unslashwillsourcethestring-funcsfile.
Usage
_unslash"pathname"##resultin
$_UNSLASH
unslash"pathname"##resultis
printed
Likemanyofthefunctionsinthisbook,unslashispairedwith
anunderscorefunction.
$unslash/usr//bin//////
/usr/bin
$_unslash//x//y//
$printf"%s\n""$_UNSLASH"
/x/y
TheScript
_unslash()
{
_UNSLASH=$1
while:
do
case$_UNSLASHin
##removetrailingslashes
*/)_UNSLASH=${_UNSLASH%/};;
##change//to/
*//*)type_gsub>/dev/null2>&1||.stringfuncs
_gsub"$_UNSLASH""//""/"
_UNSLASH=$_GSUB
;;
*)break;;
esac
done
}
unslash()
{
_unslash"$@"&&printf"%s\n""$_UNSLASH"
}
7.3checkpath—CleanUpthePATHVariable
Alongtheroutefromlogintocommandprompt,manyscriptsare
sourced.SomeofthemadddirectoriestothePATHvariable.They
rarelycheckwhetherthedirectoryisalreadyinthePATH,and
sometimestheyputthecurrentdirectoryinthePATH.Insome,thisis
deliberate;inothers,it’stheresultofcarelessscripting.
OneLinuxdistributionarrivedatthepromptwiththisPATH:
/usr//bin:/bin:/usr/bin::/usr/local/bin:/usr/X11R6/bin:/usr/ga
Thedoubleinclusionof/usr/bin(intwodifferentforms)isnot
aseriousproblem,justaminorinefficiency.However,another
directoryisimplicitlyincludedtwice.Emptyfields,createdinthis
caseby::andthefinalcolon,areinterpretedasthecurrent
directory.It’snotasseriousattheendofthePATH;anywhereelse,it
representsapotentiallydangeroussecurityhole.
Anyemptyelement(aleadingortrailingcolonortwo
successivecolons)oraperiodinyourPATHwillbeinterpretedasthe
currentdirectory.Forsecurityreasons,puttingthecurrentdirectory
inone’sPATHisnotrecommended.Ifthecurrentdirectoryisahead
of/bininyourPATH,and
arogueprogramwiththesamenameasastandardcommand
(ls,forexample)isinyourcurrentdirectory,itwouldbeexecuted,
anditcoulddamageyourfilesoryoursystem.Itisagoodideato
cleanupyourPATHinyour.profile.
HowItWorks
Byfirststandardizingtheformatofthedirectoriesin$PATH(with
unslash),checkpathcaneasilyfindduplicates.Thecurrent
directory,representedexplicitlybyadotorimplicitlybyanempty
field,willnotbeincluded.
Usage
checkpath[-v]
Inverbosemode(-v),checkpathprintsinformationaboutthe
itemsitdeletes;otherwise,itdoesitsjobsilently.
$PATH=/usr//bin::/bin:/usr/bin/:
$checkpath-v
checkpath:removing/usr/bin(already
inPATH)
$path
/usr/bin
/bin
TheScript
checkpath()
{
verbose=0
OPTIND=1
whilegetoptsvvar
do
case"$var"in
v)verbose=1;;
esac
done
##assignthedirectoriesinPATHtothepositional
parameters
oldIFS=$IFS
IFS=":"
set--$PATH
IFS=$oldIFS
newPATH=
forp##Loopthroughdirectoriesin$PATH(nowsetas
positionalparameters)
do
case$pin
""|.)continue;;##donotallowcurrent
directoryinPATH
esac
if[-d"$p"]##Isitactuallyadirectory?
then
_unslash"$p"##Removemultipleslashes
p=$_UNSLASH
case:$newPATH:in
*:"$p":*)[$verbose-ge1]&&
echo"checkpath:removing$p(alreadyin
PATH)">&2
;;
*)newPATH=${newPATH:+$newPATH:}$p;;##Add
directory
esac
else
[$verbose-ge1]&&
echo"checkpath:$pisnotadirectory;removing
itfromPATH">&2
fi
done
PATH=$newPATH
}
7.4addpath—AddDirectoriestothePATH
Variable
WhenyouaddadirectorytothePATHvariable,youwanttobesure
thatthedirectoryisnotalreadythereandthatitis,infact,a
directory.Ifyouhaveadirectoryofalternativecommands,youwill
wanttoaddittothebeginninginsteadoftheendofPATHsothat
theyareusedinplaceofthestandardversions.
HowItWorks
Theaddpathfunctionchecksthateachargumentisadirectory,and
thatthedirectoryisnotalreadyinthePATH.Iftheargumentpasses
thetests,itisaddedtotheendofPATH.The-ioptiontellsaddpath
toinsertdirectoriesatthebeginningofPATHinsteadofappending
them.
Usage
addpath[-iq]DIR...
Oneormoredirectoriesmaybespecifiedonthecommandline:
$PATH=/bin:/usr/bin
$addpath/usr/local/bin$HOME/bin
$path
/bin
/usr/bin
/usr/local/bin
/home/chris/bin
The-ioptiontellsaddpathtoplacethedirectoryatthe
beginningofPATHratherthantheend,andthe-qoptionsuppresses
theprintingoferrormessages:
$addpath-i/usr/games/chess/bin
addpath:/chessisnotadirectory
addpath:/binalreadyinpath
$addpath-q/bin##nooutput
$path
/usr/games
/bin
/usr/bin
/usr/local/bin
/home/chris/bin
Forduplicatedetectiontoworkproperly,thePATHshouldbe
standardizedwithcheckpathbeforerunningthisfunction.Ideally,
thiswillalreadyhavebeendoneaspartoftheshell’sstart-up
scripts.
TheScript
addpath()
{
##Setdefaults
prefix=0##DonotinsertatbeginningofPATH
quiet=0##Doprintinformationonbad
directories
OPTIND=1
whilegetoptsiqvar
do
case"$var"in
i)prefix=1;;
q)quiet=1;;
esac
done
shift$(($OPTIND-1))
forp##Loopthroughdirectoriesonthecommand
line
do
_unslash"$p"##removedoubleslashes
p=$_UNSLASH
case:$PATH:in
*:$p:*)[$quiet-eq0]&&echo"addpath:$p
alreadyinpath">&2
continue##Skipdirectories
alreadyinPATH
;;
esac
if[-d"$p"]
then
if[$prefix-eq1]
then
PATH="$p:$PATH"
else
PATH="$PATH:$p"
fi
else
[$quiet-eq0]&&echo"addpath:$pisnota
directory">&2
fi
done
}
7.5rmpath—RemoveOneorMoreDirectories
from$PATH
YoumightsometimeswanttoaddadirectorytothePATH,then
decidelaterthatyoudon’tneeditanymore.Removingitbyhand
canbetedious.Thisisajobforashellscript.
HowItWorks
Whenthedirectorytoberemovedisatthebeginningorendof
PATH,parameterexpansioncanremoveiteasily.Whenit’sinthe
middle,the_subfunctionfromthestring-funcslibrary(see
Chapter3)isused.
Usage
rmpathDIR...
ThecomponentsofthePATHshouldbestandardizedwith
checkpathbeforeusingrmpath.
TheScript
rmpath()#removedirectoryor
directoriesfrom$PATH
{
forpin"$@"
do
_unslash"$p"
p=$_UNSLASH
case$PATHin##Lookfor
directory....
"$p":*)PATH=${PATH#$p:};;##at
beginningofPATH
*:"$p")PATH=${PATH%:$p};;##atendof
PATH
*:"$p":*)type_sub>/dev/null2>&1||.
string-funcs
_sub"$PATH"":$p:"":"##
inthemiddle
PATH=$_SUB;;
esac
done
}
Summary
PATHisoneofthemostimportantvariablesinashellenvironment,
butitcanbeawkwardtodoanythingotherthanaddadirectoryto
it.IfthecurrentdirectoryisaddedtothePATH,whetherdeliberately
oraccidentally,therearesecurityissues.Ifanerrorismadein
typingthenameofadirectory,commandsmaynotbefound,orthe
wrongcommandsmaybeexecuted.
Thefunctionsinthischapterprovideasafewaytoaddand
removedirectoriesfromthePATH,andvalidateexistingentries.
(1) Ontario,Canada
TheY2Kbugisallbutforgotten,andtheUnix2038bugshouldbe
anon-issue(unlessyouarestillusingsoftwarethatwascompiled
morethan30yearsearlier).1Date-calculatingsoftwareinusetoday
shouldbegoodtotheyear9999,andperhapsbeyond.Unless,that
is,adjustmentshavebeenmadetothecalendartoaccommodatethe
coupleofdaysdiscrepancythatwillhavecreptinbythen.
Althoughtheuproarreachedapeakinthemonthsbefore
January1,2000,theY2Kproblemhadbeenfeltformanyyears.
Therewereprogramsthathadtodealwithpeoplebornbefore1900
andothers(oreventhesamepeople)whowouldliveintothe
twenty-firstcentury.Twodigitswasinadequate.Somesoftware
workedaroundtheproblembyadding1900whenthetwodigits
exceededacertainthreshold(say,30),andadding2000whenitwas
less.Thatstilldidn’tworkifyouweredealingbothwithpeople
bornbefore1930andpeoplewhocouldliveuntilafter2030.
Fornogoodreason,programmerscontinuedtoenterandstorea
yearastwodigits.Therewasaquipmakingtheroundsonthe
Internetinthelate1990ssaying,“Trustprogrammerstoshortenthe
year2000toY2K;that’swhatcausedtheprobleminthefirst
place.”Well,no,it’snot.TheY2Kbugwascausedbydatesbeing
storedwithonlythelasttwodigitsoftheyear.Informationwaslost
bydoingthat;noinformationislostintheabbreviationY2K.
NewYear’sEve1999slidintoNewYear’sDay2000with
hardlyaripple.WasthatbecausetheY2Kproblemhadbeen
overstated,orbecauseasterlingjobhadbeendoneinfixingall
extantsoftware?Ithinkitwasabitofboth.Ioccasionallyseedates
inthiscenturydisplayedas12/12/14,becausetheyearisstoredin
twodigits;andalotofsystemswereupgradedunnecessarily
becauseofuncertaintyaboutthebug.
Whenworkingonasystemthathasverylittlememoryor
storagespace,cramminginformationintothesmallestspace
possiblemakessense.Ididiton8-bitmachines,butwithtoday’s
computers,spaceissocheap,andcompressionsoefficient,that
there’snoexcuseforusingtwo-digityearsanymore(small,
embeddedsystemsmightbeanexception,buteventhen,there
shouldbebetterwaysofsavingspace).
Thischaptergivesyoumorethanadozenfunctionsfor
manipulatingandformattingdates.Withthemyoucanconverta
dateinalmostanyformattoanISOstandarddate(e.g.,“28
February2005”or“feb282005”to2005-02-28).Youcancalculate
howmanydayslateyourcreditorsare,orwhetheranygivendateis
avaliddate.Youcanfindthedayoftheweekonwhichyouwere
born,orhowmanydaysoldyouare.Youcanfindyesterday’sdate,
tomorrow’sdate,orthedate666daysfromnow.Youcanstore
consistentISOdatesandquicklydisplaytheminauser-friendly
format.Andyoucanusethesefunctionstobuildmanymoredate
commandsthatIdidn’thaveroomfor.
Thedate-funcsLibrary
Thescriptsinthischapterneveruseanythingbutthefullyear,
whetherasinputorstorage.Theonlyshortcuttakenisthattheyare
limitedtotheGregoriancalendar.Sincemostoftheworldhad
switchedfromtheJuliancalendarbytheendof1752,thereislittle
practicalconsequence.
8.1split_date—DivideaDateintoDay,Month,
andYear
DatesoftenarriveinaprograminaformatsuchastheInternational
Standard(ISO8601),2005-03-01,orthecommonusageof
3/7/2004(ortheunspeakable4/5/06).Formostprogramming
purposes,thisneedstobetranslatedintoeitherthreeseparate
variablesorasingleinteger(whichrequiresthethreevaluesforits
calculation).Ascriptmustbeabletosplitthedateandassignits
componentstothecorrectvariables(is3/7/2004the3rdofJulyor
the7thofMarch?).Theconversiontoanintegercomeslaterinthe
chapter.
HowItWorks
Usingacustomizedfieldseparator,IFS,split-datebreaksthedate
intothreepartsandassignseachparttoavariablespecifiedonthe
commandline.Leadingzeroesarestripped,sotheresultscanbe
usedinarithmeticexpressions.
Usage
split_date"DATE"[VAR1[VAR2
[VAR3]]]
Theelementsofthedatemaybeseparatedbywhitespace,ora
hyphen,periodorslash.Ifthedatecontainswhitespace,itmustbe
quotedonthecommandline:
$format="Day:%s\nMonth:%s\n
Year:%s\n"
$split_date"May51977"monthday
year
$printf"$format""$day""$month"
"$year"
Day:5
Month:May
Year:1977
Ifnovariablesarespecifiedonthecommandline,defaultsare
used,assigningthefirstparttoSD_YEAR,thesecondtoSD_MONTH,and
thethirdtoSD_DAY:
$split_date"1949-09-29"
$printf"$format""$SD_DAY"
"$SD_MONTH""$SD_YEAR"
Day:29
Month:9
Year:1949
Thesplit_datefunctionmakesnoassumptionsaboutthe
validityofthecomponents;theonlychangeitmakestoitsinputis
thatitremovesaleadingzero,ifthereisone,sothattheresulting
numbercanbeusedinarithmeticexpressions.
TheScript
split_date()
{
##Assigndefaultswhennovariablenamesaregivenon
thecommandline
sd_1=${2:-SD_YEAR}
sd_2=${3:-SD_MONTH}
sd_3=${4:-SD_DAY}
oldIFS=$IFS##savecurrentvalueof
fieldseparator
IFS='-/.$TAB$NL'##newvalueallowsdateto
besuppliedinotherformats
set--$1##placethedateintothe
positionalparameters
IFS=$oldIFS##restoreIFS
[$#-lt3]&&return1##Thedatemusthave3
fields
##Removeleadingzeroesandassigntovariables
eval"$sd_1=\"${1#0}\"$sd_2=\"${2#0}\"
$sd_3=\"${3#0}\""
}
8.2is_leap_year—IsThereanExtraDayThis
Year?
Somedatecalculationsneedtoknowwhetherthereare365or366
daysinanygivenyear,orwhetherFebruaryhas28or29days.
HowItWorks
Aleapyearisayeardivisibleby4butnotby100unlessitisalso
divisibleby400.ThiscalculationcanbedoneeasilyinaPOSIX
shell,butthereisafastermethodthatworkseveninaBourneshell.
Usage
is_leap_
year
[YEAR]
Ifnoyearisgiven,is_leap_yearusesdatetogetthecurrent
year.Theresultmaybederivedfromthereturncodeorfromthe
_IS_LEAP_YEARvariable:
$is_leap_year&&echoyes||echono
##assumingthecurrentyearis2005
no
$is_leap_year2004
$echo$_IS_LEAP_YEAR
1
$is_leap_year2003
$echo$_IS_LEAP_YEAR
0
TheScript
Leapyearscanbedeterminedfromthelasttwo,three,orfourdigits
ofthenumber.Forexample,ayearendingin04or08isalwaysa
leapyear.Patternmatchingreducesthenumberofpossibilitiesto
six,makingalook-uptable(thatworksinanyBourne-typeshell)
themostefficientmethod.
is_leap_year(){##USAGE:
is_leap_year[year]
ily_year=${1:-$(date+%Y)}
case$ily_yearin
*0[48]|\
*[2468][048]|\
*[13579][26]|\
*[13579][26]0|\
*[2468][048]00|\
*[13579][26]00)_IS_LEAP_YEAR=1
return0;;
*)_IS_LEAP_YEAR=0
return1;;
esac
}
Notes
Althoughslowerthantheprecedingscript,thecanonicalmethodof
checkingforaleapyearwouldhavebeenaperfectlyacceptable
scriptinthisbook:
is_leap_year(){
ily_year=${1:-`date+%Y`}
[$(($ily_year%400))-eq0-o\
$$(($ily_year%4))-eq0-a$(($ily_year%
100))-ne0$]&&{
_IS_LEAP_YEAR=1
return0
}||{
_IS_LEAP_YEAR=0
return1
}
}
8.3days_in_month—HowManyDaysHath
September?
Onemethodofdeterminingwhetheradateisvalidusesthenumber
ofdaysinthemonth.Ifthedayenteredishigherthanthenumberof
daysinthemonth,theprogramshouldrejectthatdate.
HowItWorks
Asimplelook-uptablewillgivethenumberofdaysinanymonth
exceptFebruary.ForFebruary,theyearisalsonecessary.
Usage
_days_in_month[month[year]]##
resultin_DAYS_IN_MONTH
days_in_
month
[month[year]]##resultisprinted
Ifnomonthisenteredonthecommandline,date_vars,from
thestandard-funcslibraryinChapter1,isusedtogetthecurrent
monthandyear.IfthemonthisFebruaryandnoyearisgiven,date
isusedtogetthecurrentyear.
$days_in_month##itisnow
February2005
28
$days_in_month22004
29
TheScript
_days_in_month()
{
if[-n"$1"]##Ifthere’sacommand-line
argument...
then
dim_m=$1##$1isthemonth
dim_y=$2##$2istheyear
else##Otherwiseusethecurrentdate
date_vars##setdatevariables(from
standard-funcs)
dim_y=$YEAR
dim_m=$MONTH
fi
case${dim_m#0}in
##ForallmonthsexceptFebruary,
##asimplelook-uptableisallthat’sneeded
9|4|6|11)_DAYS_IN_MONTH=30;;##30dayshath
September...
1|3|5|7|8|10|12)_DAYS_IN_MONTH=31;;
##ForFebruary,theyearisneededinorderto
check
##whetheritisaleapyear
2)is_leap_year${dim_y:-`date+%Y`}&&
_DAYS_IN_MONTH=29||_DAYS_IN_MONTH=28
;;
*)return5;;
esac
}
days_in_month()
{
_days_in_month$@&&printf%s\n$_DAYS_IN_MONTH
}
8.4JulianDates
Whatdatewillitbe45daysfromnow?Howmanydaysarethere
untilmyrentisdue?Howmanydaysoverdueispaymentforthat
invoiceIsent?HowmanydaysuntilChristmas?Theeasywayto
calculatedatessuchastheseistoconvertthemtointegers;thedates
canthenbemanipulatedbysimpleadditionandsubtraction.
HowItWorks
TheJulianDatesystemusedbyastronomerswasinventedby
JosephScaligerin1583andnamedforhisfather,JuliusCaesar
Scaliger,nottheRomanemperor(thoughIexpecttheconnection
contributedtothechoiceofname).TheJulianDateisthenumberof
dayssincenoononJanuary1,-4712,i.e.,January1,4713BC.The
timeofdayisrepresentedbyadecimalfraction.SincetheJulian
daystartsatnoon,.5ismidnight,and.75is6:00a.m.
Forourpurposes,wewantthedaynumbertorefertothe
calendarday,frommidnighttomidnight,soweusetheJulianDay
(JD)numberatnoon,whichhasatimecomponentof0.Thereare
variousformulasforcalculatingtheJDnumber;thisbookusesone
thatwaspostedtothecomp.unix.shellnewsgroupbyTapani
Tarvainen.
Usage
date2julian[YEAR-MONTH-DAY]
julian2dateJulianDayNumber
TheconversionfunctionstoandfromaJulianDayaremirror
images.ThefirsttakesanISOdate(YYYY-MM-DD),day,month,
andyear,andconvertsthemtoasingleJDinteger.Thereverse
function,julian2date,convertstheJDnumbertoanISOdate.
$date2julian1974-10-18
2442339
$julian2date2441711
1973-01-28
Boththesefunctionsarepairedwithunderscoreversions
(_date2julianand_julian2date)thatjustsetavariable,butdonot
printit.
Ifdate2julianhasnoargument,thecurrentdayisused;
julian2daterequiresanargument.
TheScript
_date2julian()
{
##Ifthere’snodateonthecommandline,use
today’sdate
case$1in
"")date_vars##Fromstandard-funcs,
Chapter1
set--$TODAY
;;
esac
##Breakthedateintoyear,monthandday
split_date"$1"d2j_yeard2j_monthd2j_day||return
2
##SinceleapyearsaddadayattheendofFebruary,
##calculationsaredonefrom1March0000(a
fictionalyear)
d2j_tmpmonth=$((12*$d2j_year+$d2j_month–3))
##IfitisnotyetMarch,theyearischangedtothe
previousyear
d2j_tmpyear=$(($d2j_tmpmonth/12))
##Thenumberofdaysfrom1March0000iscalculated
##andthenumberofdaysfrom1Jan.4713BCisadded
_DATE2JULIAN=$((
(734*$d2j_tmpmonth+15)/24-2*
$d2j_tmpyear+$d2j_tmpyear/4
-$d2j_tmpyear/100+$d2j_tmpyear/400+
$d2j_day+1721119))
}
date2julian()
{
_date2julian"$1"&&printf"%s\n""$_DATE2JULIAN"
}
#ISOdatefromJDnumber
_julian2date()
{
##Checkfornumericargument
case$1in
""|*[!0-9]*)return1;;
esac
##Toavoidusingdecimalfractions,thescriptuses
multiples.
##Ratherthanuse365.25daysperyear,1461isthe
numberofdays
##in4years;similarly,146097isthenumberof
daysin400years
j2d_tmpday=$(($1-1721119))
j2d_centuries=$(((4*$j2d_tmpday-1)/146097))
j2d_tmpday=$(($j2d_tmpday+$j2d_centuries-
$j2d_centuries/4))
j2d_year=$(((4*$j2d_tmpday-1)/1461))
j2d_tmpday=$(($j2d_tmpday-(1461*$j2d_year)/
4))
j2d_month=$(((10*$j2d_tmpday-5)/306))
j2d_day=$(($j2d_tmpday-(306*$j2d_month+5)/
10))
j2d_month=$(($j2d_month+2))
j2d_year=$(($j2d_year+$j2d_month/12))
j2d_month=$(($j2d_month%12+1))
##paddayandmonthwithzerosifnecessary
case$j2d_dayin?)j2d_day=0$j2d_day;;esac
case$j2d_monthin?)j2d_month=0$j2d_month;;esac
_JULIAN2DATE=$j2d_year-$j2d_month-$j2d_day
}
julian2date()
{
_julian2date"$1"&&printf"%s\n""$_JULIAN2DATE"
}
8.5dateshift—AddorSubtractaNumberof
Days
Thedate2julianscriptletsmeconvertadatetoaninteger.Now
youneedtoputthattouseandgetyesterday’sdate,orthedateone
weekago.
HowItWorks
ByencapsulatingtheconversiontoJulianDay,thearithmetic,and
theconversionbacktoISOdateinthedateshiftfunction,asingle
commandcanreturnthedateatanyoffsetfromanygivendate.
Usage
dateshift[YYYY-MM-DD]OFFSET
Ifadate(inISOformat)isnotenteredonthecommandline,
today’sdateisused.Therefore,toretrieveyesterday’sdate:
$dateshift-1##atthetimeof
writingthefirstEdition,itisOctober15,2004
2004-10-14
WhenisTwelfthNight?
$dateshift2004-12-25+12
2005-01-06
TheScript
_dateshift()
{
case$#in
##Ifthereisonly1argument,itisthe
offset
##sousetoday’sdate
0|1)ds_offset=${1:-0}
date_vars
ds_date=$TODAY
;;
##...otherwisethefirstargumentisthedate
)ds_date=$1
ds_offset=$2
;;
esac
while:
do
case$ds_offsetin
0*|+*)ds_offset=${ds_offset#?};;##
Removeleadingzerosorplussigns
-*)break;;##
NegativevalueisOK;exittheloop
"")ds_offset=0;break;;##Empty
offsetequals0;exitloop
*[!0-9]*)return1;;##
Containsnon-digit;returnwitherror
*)break;;##Let’s
assumeit’sOKandcontinue
esac
done
##ConverttoJulianDay
_date2julian"$ds_date"
##AddoffsetandconvertbacktoISOdate
_julian2date$(($_DATE2JULIAN+$ds_offset))
##Storeresult
_DATESHIFT=$_JULIAN2DATE
}
dateshift()
{
_dateshift"$@"&&printf"%s\n""$_DATESHIFT"
}
Notes
Itisconvenienttohaveseparatecommandsforthecommonlyused
calculations:
_yesterday()
{
_date2julian"$1"
_julian2date$(($_DATE2JULIAN-1))
_YESTERDAY=$_JULIAN2DATE
}
_tomorrow()
{
_date2julian"$1"
_julian2date$(($_DATE2JULIAN+1))
_TOMORROW=$_JULIAN2DATE
}
8.6diffdate—FindtheNumberofDaysBetween
TwoDates
Howmanydaysdoyouhaveuntilthenextdeadline?Howmany
daysarebetweenmybirthdayandChristmas?
HowItWorks
Thisisanotherencapsulationofconversionandarithmetic,using
twodates.
Usage
diffdate
YYYY-MM-DD[YYYY-MM-DD]
Ifonlyonedateisgiven,thecurrentdateisusedasthefirst
date:
$diffdate2004-12-25##todayis
October15,2004
71
Iftheseconddateisearlierthanthefirst,theresultwillbe
negative:
$diffdate2005-03-221998-10-18
-2347
TheScript
Thisscriptsimplyconvertsthetwodates(oronedateandtoday’s
date)andsubtractsonefromtheother.
_diffdate()
{
case$#in
##Ifthere’sonlyoneargument,usetoday’s
date
1)_date2julian$1
dd2=$_DATE2JULIAN
_date2julian
dd1=$_DATE2JULIAN
;;
2)_date2julian"$1"
dd1=$_DATE2JULIAN
_date2julian"$2"
dd2=$_DATE2JULIAN
;;
esac
_DIFFDATE=$(($dd2-$dd1))
}
diffdate()
{
_diffdate"$@"&&printf"%s\n""$_DIFFDATE"
}
8.7day_of_week—FindtheDayoftheWeek
forAnyDate
Youneedtoknowwhetheracertaindateisabusinessday,or
whetheritisontheweekend.
HowItWorks
ThisisanotherusefortheJulianDay.Ifyouadd1totheJDand
divideby7,theremainderisthedayoftheweek,countingfrom
Sundayas0.Alook-uptableconvertsthenumbertoaname.
Usage
day_of_
week
[YYYY-MM-DD]
daynameN
TofindwhichdayoftheweekChristmasison,usethis:
$day_of_week2005-12-25
0
ChristmasDay,2005fallsonaSunday.Toconvertthenumber
tothenameoftheday,daynameusesasimplelook-uptable:
$dayname0
Sunday
Thedaynamefunctionwillalsocompletethenameofthedayif
yougiveitanabbreviation:
$daynamewed
Wednesday
Ifday_of_weekiscalledwithoutanargument,itusesthecurrent
day’sdate.
TheScript
_day_of_week()
{
_date2julian"$1"
_DAY_OF_WEEK=$((($_DATE2JULIAN+1)%7))
}
day_of_week()
{
_day_of_week"$1"&&printf"%s\n""$_DAY_OF_WEEK"
}
##Daynameacceptseither0or7for
Sunday,2-6fortheotherdays
##orchecksagainstthefirstthree
letters,inupperorlowercase
_dayname()
{
case${1}in
0|7|[Ss][Uu][Nn]*)_DAYNAME=Sunday;;
1|[Mm][Oo][nN]*)_DAYNAME=Monday;;
2|[Tt][Uu][Ee]*)_DAYNAME=Tuesday;;
3|[Ww][Ee][Dd]*)_DAYNAME=Wednesday;;
4|[Tt][Hh][Uu]*)_DAYNAME=Thursday;;
5|[Ff][Rr][Ii]*)_DAYNAME=Friday;;
6|[Ss][Aa][Tt]*)_DAYNAME=Saturday;;
*)return5;;##Nomatch;returnanerror
esac
}
dayname()
{
_dayname"$@"&&printf"%s\n""$_DAYNAME"
}
8.8display_date—ShowaDateinTextFormat
TheISOdateformatisgreatforcomputers,butit’snotaseasyfor
peopletoread.Weneedawaytodisplaydatesinamoreuserfriendlyway.
HowItWorks
Look-uptablesconvertthedayoftheweekandthemonthtotheir
respectivenames.Theusercansupplydifferentstringstovarythe
formatused.
Usage
display_
date
[-fFMT][YYYY-MM-DD]
Ifnoformatissupplied,thedefaultisused:
$display_date2005-02-14
Wednesday,14February2005
Thereareonlyfourformatstringsdefinedatthemoment(
WMdy,dMy,Mdy,andWdMy),butyoucaneasilyaddmore.Here
areexamplesofallfourformats:
$forfmtinWMdydMyMdyWdMy
>do
>date=$(($RANDOM%100+1950))$(($RANDOM%12))-$(($RANDOM%28))
>display_date-f"$fmt""$date"
>done
Thursday,July12,1998
14November1964
January21,2009
Monday,18January2018
TheScript
display_date()
{
dd_fmt=WdMy##Defaultformat
##Parsecommand-lineoptionsforformatstring
OPTIND=1
whilegetoptsf:var
do
case$varin
f)dd_fmt=$OPTARG;;
esac
done
shift$(($OPTIND-1))
##Ifthereisnodatesupplied,usetoday’sdate
case$1in
"")date_vars##Functionfromstandard-funcs
inChapter1
set--$TODAY
;;
esac
split_date"$1"dd_yeardd_monthdd_day||return2
##Lookuplongnamesfordayandmonth
_day_of_week"$1"
_dayname$_DAY_OF_WEEK
_monthname$dd_month
##Printdateaccordingtoformatsupplied
case$dd_fmtin
WMdy)printf"%s,%s%d,%d\n""$_DAYNAME"
"$_MONTHNAME"\
"$dd_day""$dd_year";;
dMy)printf"%d%s%d\n""$dd_day"
"$_MONTHNAME""$dd_year";;
Mdy)printf"%s%d,%d\n""$_MONTHNAME"
"$dd_day""$dd_year";;
WdMy|*)printf"%s,%d%s%d\n""$_DAYNAME"
"$dd_day"\
"$_MONTHNAME""$dd_year";;
esac
}
##Setthemonthnumberfrom1-or2digitnumber,orthename
_monthnum()
{
case${1#0}in
1|[Jj][aA][nN]*)_MONTHNUM=1;;
2|[Ff][Ee][Bb]*)_MONTHNUM=2;;
3|[Mm][Aa][Rr]*)_MONTHNUM=3;;
4|[Aa][Pp][Rr]*)_MONTHNUM=4;;
5|[Mm][Aa][Yy]*)_MONTHNUM=5;;
6|[Jj][Uu][Nn]*)_MONTHNUM=6;;
7|[Jj][Uu][Ll]*)_MONTHNUM=7;;
8|[Aa][Uu][Gg]*)_MONTHNUM=8;;
9|[Ss][Ee][Pp]*)_MONTHNUM=9;;
10|[Oo][Cc][Tt]*)_MONTHNUM=10;;
11|[Nn][Oo][Vv]*)_MONTHNUM=11;;
12|[Dd][Ee][Cc]*)_MONTHNUM=12;;
*)return5;;
esac
}
monthnum()
{
_monthnum"$@"&&printf"%s\n""$_MONTHNUM"
}
##Setthemonthnamefrom1-or2digitnumber,orthename
_monthname()
{
case${1#0}in
1|[Jj][aA][nN])_MONTHNAME=January;;
2|[Ff][Ee][Bb])_MONTHNAME=February;;
3|[Mm][Aa][Rr])_MONTHNAME=March;;
4|[Aa][Pp][Rr])_MONTHNAME=April;;
5|[Mm][Aa][Yy])_MONTHNAME=May;;
6|[Jj][Uu][Nn])_MONTHNAME=June;;
7|[Jj][Uu][Ll])_MONTHNAME=July;;
8|[Aa][Uu][Gg])_MONTHNAME=August;;
9|[Ss][Ee][Pp])_MONTHNAME=September;;
10|[Oo][Cc][Tt])_MONTHNAME=October;;
11|[Nn][Oo][Vv])_MONTHNAME=November;;
12|[Dd][Ee][Cc])_MONTHNAME=December;;
*)return5;;
esac
}
monthname()
{
_monthname"$@"&&printf"%s\n""${_MONTHNAME}"
}
8.9parse_date—DecipherVariousFormsof
DateString
Datescomeinmanydifferentformats,somestraightforward,some
ambiguous.Someareeasytoparse,othersarenot.Someputthe
monthbeforethedate,someputitafter.Someusenumbersforthe
month,someusethenameofthemonth.Someseparatethe
componentswithspaces,somewithslashes.Outofthesemany
formats,isthereawaytoextractacoherentdate?
HowItWorks
Designedtogivethewidestpossiblelatitudeindateentry,
parse_datecanbetoldtoexpectaspecificformat,oritcanchewon
theinputand,withabitofluck,spitoutavalidISOdate.Thethree
optionstellparse_datetointerpretthefieldsasYMD,DMY,or
MDY.Ifnooptionisgiven,thescriptwillattempttofigureout
whatismeant.
Usage
parse_date[-eiu]DATE
Theoptionssettheorderthatparse_dateassignsthefields:-e
usesday-month-year,-uusesmonth-day-year,and-iusesyearmonth-day(thinkEnglish,US,andInternational).
Themonthmaybeenteredasanumberfrom1to12,orasthe
nameofthemonth;thefieldsmaybeseparatedbywhitespace,
periods,hyphens,orslashes.TheresultisprintedasanISOdate,
year-month-day:
$parse_date-e12.4.2001
2001-04-12
$parse_date-u12.4.2001
2001-12-04
$parse_date12.apr.2001
2001-04-12
Invaliddatesarecaught,andanerrorisreturned:
$parse_date-u2004-12-10;echo$?
2
$parse_date-i12.4.2001;echo$?
2
Thereareafewshortcuts.Today,yesterday,andtomorrowcan
beusedinsteadoftheactualdates;thesecanberepresentedbya
period,hyphen,andplussign,respectively:
$parse_date.
2004-10-15
$parse_date+
2004-10-16
$parse_date
2004-10-14
Adate30daysfromnowmaybeenteredas+30;oneweekago
canbeenteredas-7:
$parse_date+30
2004-11-14
$parse_date-7
2004-10-08
Ambiguousdatesreturnanerror(butanoptioncanremovethe
ambiguity):
$parse_date2.3.2001||echo
ambigous>&2
ambigous
$parse_date-e2.3.2001||echo
ambigous
2001-03-02
$parse_date-u2.3.2001||echo
ambigous
2001-02-03
TheScript
_parse_date()
{
##Clearvariables
_PARSE_DATE=
pd_DMY=
pd_day=
pd_month=
pd_year=
##Ifnodateissupplied,readonefromthe
standardinput
case$1in
"")[-t0]&&printf"Date:">&2##Prompt
onlyifconnectedtoaterminal
readpd_date
set--$pd_date
;;
esac
##Acceptyesterday,todayandtomorrowasvalid
dates
case$1in
yes*|-)
_yesterday&&_PARSE_DATE=$_YESTERDAY
return
;;
tom*|+)
_tomorrow&&_PARSE_DATE=$_TOMORROW
return
;;
today|.)
date_vars&&_PARSE_DATE=$TODAY
return
;;
today*|\
.[-+1-9]*|\
[-+][1-9]*)
pd_=${1#today}
pd_=${pd_#[-+]}
_dateshift$pd_&&
_PARSE_DATE=$_DATESHIFT
return
;;
esac
##Parsecommand-lineoptionsfordateformat
OPTIND=1
whilegetoptseiuvar
do
case$varin
e)pd_DMY=dmy;;
i)pd_DMY=ymd;;
u)pd_DMY=mdy;;
esac
done
shift$(($OPTIND-1))
##Splitdateintothepositionalparameters
oldIFS=$IFS
IFS='/-.
$TAB$NL'
set--$*
IFS=$oldIFS
##Ifdateisincomplete,usetoday’sinformationto
completeit
if[$#-lt3]
then
date_vars
case$#in
1)set--$1$MONTH$YEAR;;
2)set--$1$2$YEAR;;
esac
fi
case$pd_DMYin
##Interpretdateaccordingtoformatifonehas
beendefined
dmy)pd_day=${1#0};pd_month=${2#0};pd_year=$3
;;
mdy)pd_day=${2#0};pd_month=${1#0};pd_year=$3
;;
ymd)pd_day=${3#0};pd_month=${2#0};pd_year=$1
;;
##Otherwisemakeaneducatedguess
*)case$1--$2-$3in
[0-9][0-9][0-9][0-9]*-*)
pd_year=$1
pd_month=$2
pd_day=$3
;;
*--[0-9][0-9][0-9][0-9]*-*)##strange
place
pd_year=$2
_parse_dm$1$3
;;
*-[0-9][0-9][0-9][0-9]*)
pd_year=$3
_parse_dm$1$2
;;
*)return5;;
esac
;;
esac
##Ifnecessary,convertmonthnametonumber
case$pd_monthin
[JjFfMmAaSsOoNnDd]*)_monthnum"$pd_month"||
return4
pd_month=$_MONTHNUM
;;
*[!0-9]*)return3;;
esac
##Quickchecktoeliminateinvaliddayormonth
["${pd_month:-99}"-gt12-o"${pd_day:-99}"-gt31
]&&return2
##Checkforvaliddate,andpaddayandmonthif
necessary
_days_in_month$pd_month$pd_year
case$pd_dayin?)pd_day=0$pd_day;;esac
case$pd_monthin?)pd_month=0$pd_month;;esac
[${pd_day#0}-le$_DAYS_IN_MONTH]&&
_PARSE_DATE="$pd_year-$pd_month$pd_day"
}
parse_date()
{
_parse_date"$@"&&printf"%s\n"
"$_PARSE_DATE"
}
##Calledby_parse_datetodeterminewhichargument
isthemonth
##andwhichistheday
_parse_dm()
{
##functionrequires2arguments;morewillbe
ignored
[$#-lt2]&&return1
##ifeitherargumentbeginswiththefirst
letterofamonth
##it’samonth;theotherargumentistheday
case$1in
[JjFfMmAaSsOoNnDd]*)
pd_month=$1
pd_day=$2
return
;;
esac
case$2in
[JjFfMmAaSsOoNnDd]*)
pd_month=$2
pd_day=$1
return
;;
esac
##returnerrorifeitherargcontainsnon-numbers
case$1$2in*[!0-9]*)return2;;esac
##ifeitherargumentisgreaterthan12,itisthe
day
if[$1-gt12]
then
pd_day=$1
pd_month=$2
elif[${2:-0}-gt12]
then
pd_day=$2
pd_month=$1
else
pd_day=$1
pd_month=$2
return1##ambiguous
fi
}
8.10valid_date—WhereWasIonNovember
31st?
Whenyoualreadyhaveawell-formeddate,parse_dateisoverkill
forverification.Asimplerscriptwillsuffice.
HowItWorks
IfyouconvertadatetoaJulianDay,thenconvertitback,the
valueswillnotbethesameifthedateisnotvalid:
$julian2date$(date2julian2004-12-32)
2005-01-01
Thisfunctiondoesjustthat,andfailsifthedatesdonotmatch.
Usage
valid_
date
YEAR-MONTH-DAY
Thereareonly30daysinApril,soApril31willnotbevalid:
$valid_date2005-04-31&&echoOK||
echoInvaliddate
Invaliddate
$valid_date2005-04-30&&echoOK||
echoInvaliddate
OK
TheScript
valid_date()
{
_date2julian"$1"||return8
_julian2date$_DATE2JULIAN||return7
[$_JULIAN2DATE=$1]
}
Summary
Manyofthefunctionspresentedherecanbeachievedwiththe
GNUversionofthedatecommand.Others,suchasparse_date,
havemoreflexibility.Thesefunctionswilloftenbefasterthandate,
butnotalwaysbymuch.Theydospeedupscriptwritingbytucking
manydetailsawaybehindthescenes.
Manymoredatefunctionscouldbeincluded,somethathaven’t
yetbeenseparatedintotheirownfunctionsbecausetheyaresmall
additionstoscriptsalreadyhere,andtheycaneasilybeenteredinlinewiththerestofascript.Someareherebecausetheycouldbe
usedalot,orbecausetheyanswersomenewsgroupFAQs.
Anexamplethatfitsintobothcategoriesisyesterday.You
couldhaveacronjobthatrunseverynightatmidnightandarchives
certainfiles.Youmightwanttodatethemwiththeprecedingday’s
date.Havingaone-wordcommandmadethescripteasiertowrite,
andtheresulteasiertoread.
Ihavethoughtaboutwritingascriptthatwouldaccepta
formattingstringforprintingthedate,adisplay_dateonsteroids,
butitwouldbefarmoreworktowritethanitwarrants.Youcanuse
GNUdatewhenyouneedthatfacility.
Footnotes
1 OnUnix,thesystemtimeisstoredassecondssincetheepoch(1January1970)ina32bitinteger.Themaximumcapacityofsuchanintegerwillbereachedon19January
2038.Thiswillbe(andalreadyisbeing)solvedbyusinga64-bitinteger,whetherona
64-bitcomputerwithanativeintegerofthatsize,orwiththefieldbeingdefinedasa64bitintegerona32-bitcomputer.Foramorecompletediscussion,seeTheYear2038
ProblembyRogerM.Wilcoxathttp://pw2.netcom.com/ rogermw/Y2038.html.
Chapter9:GoodHousekeeping:
MonitoringandTidyingUpFile
Systems
(1) Ontario,Canada
Whetheryouarethesystemadministratororjustauser,whether
youhaveyourownboxoranaccountonaremotemachine,sooner
orlateryouwillneedtodosomeclean-upwork.Youwillwantto
removeduplicatefilesandcleanoutemptyfiles.Symboliclinksare
sometimeslefthangingwhenafileisremoved,soyouwould
probablyliketodeletethose.And,ofcourse,youwillwanttokeep
aneyeonyourharddrives’levelsofusage.
9.1dfcmp—NotifyUserofMajorChangesin
DiskUsage
Unixsystemsdonottakekindlytorunningoutofdiskspace.Ihave
seenacomputerreducedtoajabberingidiotwhentherewasno
roomleftontheharddrivetowriteamessagetoalogfile.This
generatedanerror,whichthesystemtriedtowritetoalogfile.
Sincetherewasnospaceonthe/var/logpartition,thesystem
generatedanerrorthatittriedtowritetoalogfile.Therewasno
spaceonthepartition,sothesystemgeneratedanerror,whichit
triedtowritetoalogfile.Thepartitionwasfull,so...
Asotherprocessestriedtowritetoalogfile,moreandmore
errorsweregenerated.Themachineslowedtoacrawlwhile
spewinghundreds(ormaybethousands)oferrormessagesper
secondtotheconsole.Thecomputerquicklybecameunusable.
Ineededsomethingtowarnmewhendrivesweregettingfull,or
whentherewasanymajorchangeindiskusage.
HowItWorks
Everynight,acrontabentryrunsdfcmp,whichusesthedfcommand
tosavethecurrentstateofthedrives(asapercentageofspaceused)
toafile.Thisfileiscomparedtothepreviousday’sfile,aswellas
thepreviousweek’sandpreviousmonth’sfiles.Ifthedifference
exceeds10percentagepoints,areportissent.Ifthespaceusedis
greaterthan90%,onlya1%changeisneededtosetoffthealarm.
Theamountofchangeneededtogenerateawarningcanbe
adjustedbyaconfigurationfileorbycommand-lineoptions.
Usage
dfcmp[-dNN][-hNN][-pNN][-f][-cCFG]
Thed,h,andpoptionssetthepercentagesofdifferencerequired
beforeareportisgenerated,andthepointatwhichtheamountused
changes.Toreceiveareportifthedifferenceisgreaterthan5%,and
whenitexceeds2%whenthefilesystemismorethan80%full:
dfcmp-d5-h2-p80
The-coptiontellsdfcmptouseaconfigurationfileotherthan
thedefault,$HOME/.config/dfcmp.cfg.
TheScript
Tofindthedateonemonthago,the_last_monthfunctionassumes
thatdate_varshasbeencalled,andthecurrentdateisthereforein
$_DAY,$_MONTH,and$_YEAR.
_last_month()
{
lm_day=0$_DAY
lm_month=$_MONTH
lm_year=$YEAR
case$_MONTHin
##Ifit’sJanuary,useDecemberoftheprevious
year
1)lm_year=$(($lm_year-1))
lm_month=12
;;
##otherwisejustsubtractonefromthemonth
*)lm_year=$YEAR
lm_month=0$(($lm_month-1))
;;
esac
##Checkthatthedayisnogreaterthanthedaysin
lastmonth
##andsettolastdayofthemonthifitis
if["${lm_day#0}"-gt28]
then
_days_in_month$lm_month$lm_year
["${lm_day#0}"-gt"$_DAYS_IN_MONTH"]&&
lm_day=$_DAYS_IN_MONTH
fi
_LAST_MONTH=$lm_year-${lm_month#${lm_month%??}}${lm_day#${lm_day%??}}
}
Tocomparetwodf_filesandprintmajordifferences,the
cmp_dffunctionusestheUnixjoincommandtomergethetwofiles
onthefirstfield.Eachlinethencontainsafile-systemmountpoint
anditspercentageusedfromeachfile.Ifthedifferenceisgreater
thantheconfiguredthreshold,theinformationisprinted.
cmp_df()
{
[-f"$1"]&&[-f"$2"]&&
join-t"$TAB"$1$2|
whilereadfstodayyesterday
do
notify=0
[$today-eq$yesterday]&&continue
diff=$(($today-$yesterday))
if[$today-ge$high_point]
then
[${diff#-}-ge$high_diff]&&notify=1
elif[${diff#-}-ge$min_diff]
then
notify=1
fi
if[$notify-gt0]
then
printf"%20s%9d%%%7d%%(%d)\n"$fs$today
$yesterday$diff
fi
done
}
progname=${0##*/}##Extractthename
ofthescript
case$prognamein
-sh)shx=-sh;;
*)shx=;;
esac
.date-funcs$shx
df_dir=/var/log/df
[-d"$df_dir"]||mkdir"$df_dir"||
die5"Couldnotcreatedirectory,
$df_dir"
##Defaults,maybeoverriddenby
configfileoroptions
min_diff=10##levelofchange
neededtogeneratereport
high_diff=1##levelofchangeto
reportwhendiskisnearlyfull
high_point=90##levelatwhich
$high_diffisusedinsteadof$min_diff
configfile=$HOME/.config/dfcmp.cfg##
defaultconfigfile
mkfile=0##donotre-create
today’sfileifitalreadyexists
whilegetoptsfc:d:h:p:var
do
case$varin
c)configfile=$OPTARG;;
d)min_diff=$OPTARG;;
h)high_diff=$OPTARG;;
p)high_point=$OPTARG;;
f)mkfile=1;;##forcecreationoftoday’sfile
esac
done
shift$(($OPTIND-1))
##checkforconfigfile,andsourceit
ifitexists
[-f"$configfile"]&&."$configfile"
##settoday’sdatevariables
date_vars
##setfilenames
df_file=$df_dir/df_$TODAY
_yesterday$TODAY
df_yesterfile=$df_dir/df_$_YESTERDAY##yesterday
_dateshift$TODAY-7||die2
df_lastweekfile=$df_dir/df_$_DATESHIFT##lastweek
_last_month||die2
df_lastmonthfile=$df_dir/df_$_LAST_MONTH##lastmonth
##createtoday’sfileifitdoesn’t
exist(orif-fisselected)
[-f"$df_file"]||mkfile=1
[$mkfile-eq1]&&
df|{read##discardheader
whilereadfsbuapcm
do
case$fsin
##skipsambaandNFSmounts,proc,etc.
//*|*:*)continue;;
*/*);;##regularfilesystem
*)continue;;##unknown;skipit
esac
##whenlineissplitduetolongfilesystem
name,readnextline
case$bin"")readbuapcm;;esac
printf"%s\t%d\n"$m"${pc%\%}"
done
}|sort>"$df_file"
##Comparetoday’sfilewitheachof
thethreeoldfiles
setYESTERDAY"1WEEKAGO""1MONTH
AGO"
forfilein$df_yesterfile
$df_lastweekfile$df_lastmonthfile
do
result=`cmp_df$df_file$file`
if[-n"$result"]
then
printf"%20s%10s%10s%s\n"FILESYSTEMTODAY"$1"
DIFF
printf"%s\n""$result"
fi
shift
done
9.2symfix—RemoveBrokenSymbolicLinks
Becausesymboliclinksrefertofilesbyname,theycanbebroken
bytheremovalofthefiletheypointto.Brokenlinksdon’treally
causeanyproblems,buttheycanmakeadirectorylookuntidy.
Unlessyouwanttokeepabrokenlinkasaremindertocreateits
target(or,perhaps,watchforthecreationofthetarget),whynotget
ridofthem?
HowItWorks
Usingalonglisting(ls-l)pipedthroughgrep^l,thesymlinked
filesandtheirtargetsareextracted.Ifthetargetdoesn’texist,the
linkisremoved.
Usage
symfix[OPTIONS][FILE...|DIR...]
Bydefault,symfixremovesallthebrokenlinksinthecurrent
directory.Ifthefirstargumentisadirectory,allargumentsare
assumedtobedirectories;eachoneisenteredinturn,andallbroken
linksareremoved.Otherwise,anyargumentsaretakenas
filenames,andanythatarebrokenlinkswillberemoved.
Threeoptionsareacceptedbysymfix:-i,-v,and-R(interactive,
verbose,andrecursive,respectively).Ininteractivemode,theuser
ispromptedandmaydeleteorleaveeachbrokenlink.Verbose
modeprintsthenamesofalldirectoriesitenters,andalllinks
removed.Recursivemodedeletesallbrokenlinksinthedirectories
listedonthecommandlineandalltheirdescendants;ifno
directoriesareonthecommandline,brokenlinksareremovedfrom
thecurrentdirectoryandtheentiretreebeneathit.Inrecursive
mode,afilepatternisonlyeffectiveonthecurrentdirectory,and
onlyifnodirectoriesaregiven.
TheScript
Thefix_pwdfunctionprocessesbrokensymlinksinthecurrent
directory,promptingtheuserwhenininteractivemode.
fix_pwd()
{
ls-l"$@"|grep"^l"|
whileread-rpermslinksownergroupsizemonthday
timefilexlink
do
link=${file#*-\>}
file=${file%-\>*}
[-e"$link"]||{
if[$interactive-ge1]
then
printf"Remove%s(%s)(Y/n/q)?""$file"
"$link"
get_key</dev/tty
printf"\n"
case$_KEYin
y|Y|"")ok=1;;
q|Q)break;;
*)ok=0;;
esac
fi
[$verbose-ge1]&&
printf"Brokenlink:'%s';removing%s\n"
"$link""$file">&2
["$ok"-eq1]&&rm"$file"
}
done
}
progname=${0##*/}##extractnameof
script
##Defaultsettings
interactive=0
recurse=0
verbose=0
R_opt=
ok=1
opts=ivR
.standard-funcs##loadfunctions
do
case$varin
i)interactive=1
R_opt="$R_opt-i";;
R)recurse=1;;
v)verbose=$(($verbose+1))
R_opt="$R_opt-v";;
esac
done
shift$(($OPTIND-1))
#######is_dirfunctionisinstandard-funcs#######
##Ifadirectoryisgivenonthe
commandline,enterit
##(orloopthrougheachofthem,if
thereismorethanone)
##Inrecursivemode,is_diriscalled
tocheck
##whetherthereareanysubdirectories
inthecurrentdirectoryif[-d"$1"]
then
fordir
do
(
[-L"$dir"]&&continue##donotdescend
symlinkeddirectories
cd"$dir"||continue
[$verbose-ge1]&&printf"Directory:%s\n"
"$PWD"
fix_pwd
[$recurse-eq1]&&is_dir./*/&&eval"$0-R
$R_opt./*/"
)
done
else
fix_pwd"$@"
[$recurse-eq1]&&is_dir./*/&&eval"$0-R$R_opt
./*/"
:##returnsuccessfullyevenif[$recurse-eq1]
fails
fi
9.3sym2file—ConvertsSymbolicLinksto
RegularFiles
AsIwritethisbook,Ihaveaseparatedirectoryforeachchapter.In
each,Ihavesymboliclinkstothatchapter’sscripts,whichIkeepin
mymainscriptsdirectory.WhenIfinishthedraftofachapter,I
wanttokeepeachscriptasIuseditinthechapter,andnothaveit
changewheneverItinkerwiththescript.Todothat,Ineededto
converteachsymlinktoaregularfile.
HowItWorks
Aswithsymfix,theoutputofls-lisparsedtoextractthefileand
itstarget;thefileisremoved,andthetargetiscopiedintoitsplace.
Usage
sym2file[OPTION][FILE...]
Withtheinteractiveoption,-i,sym2filepromptsbeforedoing
theconversion.Withtheverboseoption,alloperationsareechoed
tothescreen.
NotethatalthoughthescriptexaminestheBASH_VERSION
variable,itdoesnotrequirebashtorun.Ifthefileisbeingexecuted
bybash(version2orlater)amoreefficientget_keyfunctionis
used.
TheScript
Ratherthanusethegenericget_keyfunctionfromstandard-funcs
(seeChapter1),thecapabilityofbash’sbuilt-inreadcommandto
returnafteraspecifiednumberofcharactersisusedinsteadofstty
anddd—ifacapableversionofbashisexecutingthescript.
case$BASH_VERSIONin
[2-9]*|[1-9][0-9]*)##bash2orlater
get_key()
{
read-sn1-u0${1:-_KEY}
}
;;
*)[-t0]&&_STTY=`stty-g`
get_key()
{
[-t0]&&stty-echo-icanon
_KEY=`ddbs=1count=12>/dev/null
</dev/tty`
[-n"$1"]&&eval"$1=\"\$_KEY\""
[-t0]&&stty"$_STTY"
[-n"$_KEY"]
}
;;
esac
progname=${0##*/}##Extractnameof
script
##Defaults
interactive=0
verbose=0
opts=iv
do
case$varin
i)interactive=1;;
esac
done
shift$(($OPTIND-1))
##
ls-l"$@"|grep"^l"|
whilereadpermslinksownergroupsizemonthdaytime
file
do
link=${file#*-\>}
file=${file%-\>*}
##Ifthelinkdoesnotpointtoafile,skipit
[-f"$link"]||{
[$verbose-ge1]&&
printf"'%s'isnotafile;skipping\n"
"$link">&2
continue
}
ok=1
##Ininteractivemode,prompttheuser
foradecision
if[$interactive-ge1]
then
printf"Copy%sto%s(Y/n/q)?""$link""$file"
get_key</dev/tty
printf"\n"
case$_KEYin
y|Y|"")ok=1;;
q|Q)break;;
*)ok=0;;
esac
fi
##Inbatchmode,orwhentheuserhas
giventhegreenlight,
##deletethesymlinkandcopythefile
intoitsplace.
if[$ok-ge1]
then
rm"$file"&&[$verbose-ge1]&&printf"Deleted:
%s\n""$file"
cp-p"$link""$file"&&[$verbose-ge1]&&
printf"Copied:'%s'->
'%s'\n""$link""$file"||
touch"$file"
fi
done
9.4zrm—RemoveEmptyFiles
Sometimes,emptyfilesexistforareason,perhapstochangea
program’sbehaviorbytheexistenceornonexistenceofthefile.
Usually,however,theyarenotwanted;theyhavebeencreated
accidentallyorinerror,andyouwillwanttodeletethem.
HowItWorks
Ratherthangettingafile’ssizebyparsingtheoutputofls-l,we
canusethetestcommand(whichisasynonymfor[).Ithasthe-s
operatorthatreturnstrueifthefileexistsandisnotempty.A
secondtestensuresthatthefileactuallyexistsbeforetryingto
removeit.
Usage
zrm
[OPTION][FILE...]
Thethreeoptionstozrm,-f,-i,and-v,arepasseddirectlytorm
asoptionstodothefollowing:forceremovalwithoutpromptingthe
userwhenthefileisnotwritable(butthedirectoryownershipand
permissionsallowit),prompttheuserbeforeremovingafile,and
explainwhatisbeingdone.The-voptionisnotavailableonall
versionsofrm.
TheScript
opts=
whilegetoptsfivvar
do
case$varin
f)opts="$opts-f";;
i)opts="$opts-i";;
v)opts="$opts-v";;
esac
done
shift$(($OPTIND-1))
##Ifnofilesarespecifiedonthe
commandline,
##checkallfilesinthecurrent
directory
[$#-eq0]&&set--*
##Examineeachfile,anddeleteitif
empty
forf
do
[-s"$f"]&&continue##fileexists,butisnot
empty
[-f"$f"]&&rm${opts:--f}"$f"
done
9.5undup—RemoveDuplicateFiles
Theymaybebackupcopies,theymaybethesamefiledownloaded
underdifferentnames,butsoonerorlateryouwillwindupwitha
directorycontainingduplicatefiles.Unlesstheyarelinked,they
representspacethatcanbefreed.Doingitbyhandisatedious
chore;cantheduplicatesberemovedwithascript?
HowItWorks
Thefilesaresortedbysize,usingtheoutputofls-l;whentwo
successivefilesarethesamesize,theyarecompared(usingcmp).If
thefilesareidentical,theuserispromptedwithinformationabout
thefilesandgiventheoptionsofdeletingeithercopy,symlinking
onetotheother,ordoingnothing.
Usage
undup[-z][DIR]
Ifnodirectoryisgiven,thecurrentdirectoryissearched.When
aduplicatepairisfound,informationisdisplayed,andtheuseris
askedtopressanumberfrom1to5:
$undup.
1.DeleteMinder.htm
2.DeleteMinder.html
3.Make'Minder.htm'asymlinkto'Minder.html'
4.Make'Minder.html'asymlinkto'Minder.htm'
5.Ignore
ASCIIEnglishtext:5240bytes:_
Press1,andMinder.htmwillbedeleted;press2,and
Minder.htmlisgone;press3or4,andasymboliclinkwillbe
created;press5,andyouwillproceedtothenextduplicate,ifany.
The-zoptionuseszrmtocleanoutemptyfilesbeforelooking
forduplicates.
TheScript
Theeq_queryfunctiontakestwofilenamesasargumentsandasks
theuserwhethertodeleteoneortheother,turnoneintoasymbolic
link(toretainbothnames,butreducethespaceused),ortoignore
theduplication.
eq_query()
{
[$#-ne2]&&return5
eq_1=$1
eq_2=$2
filetype=`file$eq_1`
printf"\n"
printf"\t%d.Delete%s\n"1"$eq_1"2"$eq_2"
printf"\t%s\n""3.Make'$eq_1'asymlinkto
'$eq_2'"\
"4.Make'$eq_2'asymlinkto
'$eq_1'"\
"5.Ignore"
printf"\n\t%s:%dbytes:""${filetype#*:}"$size
get_key</dev/tty
printf"\n"
case$_KEYin
1)rm"$eq_1"&&printf"\n\tRemoved$eq_1\n\n"
;;
2)rm"$eq_2"&&printf"\n\tRemoved$eq_2\n\n"
;;
3)ln-sf"$eq_2""$eq_1"&&{printf"\n";ls
-l"$eq_1";};;
4)ln-sf"$eq_1""$eq_2"&&{printf"\n";ls
-l"$eq_2";};;
q)break3;;
*)return0;;
esac
_EQ_QUERY=$_KEY
}
Filesinthecurrentdirectoryaresortedbysize,andfilesthatare
thesamesizeareexaminedbycmp;ifthefilesareidentical,
eq_queryiscalledtoasktheusertodeterminedispositionofthe
files.
rm_dups()
{
ls-l"$@"|sort-k5n|{
read-rperms_links_owner_group_size_
month_day_time_file_
whileread-rpermslinksownergroupsize
monthdaytimefile
do
[${verbose:-0}-ge1]&&printf"%10s
%s\n"$size"$file">&2
[-f"$file"]||continue
[${size:--0}-eq${size_:--1}]&&
cmp$file$file_>/dev/null&&{
[${verbose:-0}-ge1]&&printf
"%10s%s\n"$size_"$file_">&2
eq_query$file$file_
}
case$_EQ_QUERYin
1|3)continue;;
esac
size_=$size
file_=$file
done
}
}
Themainprogramloopsthroughthelistofdirectoriesonthe
commandline(usingthecurrentdirectoryiftherearenone),and
callsrm_dupsforeachone.
.standard-funcs##Loadthestandard
functions
zrm=0
TMOUT=60
opts=zv
whilegetopts$optsopt
do
case$optin
v)verbose=$((${verbose:-0}+1));;
z)zrm=1;;
esac
done
shift$(($OPTIND-1))
##Resetttyonexit
trapcleanupEXIT
set--"${@:-.}"
fordir
do
(
cd"${dir:-.}"||continue
:${PWD:=`pwd`}
[$zrm-ge1]&&zrm*
rm_dups
)
done
9.6lsr—ListtheMostRecent(orOldest)Files
inaDirectory
Ioftenwanttoviewthemostrecentfilesinadirectory.Itmaybe
mydownloaddirectorybecauseIdon’trememberthenameofthat
fileIjustpulledofftheWeb,oritmaybebecauseIhaveforgotten
thenameIusedforthescriptIwroteafewhoursago(don’tlaugh;
ithappens).
HowItWorks
Lazinessisthemotherofinvention,andthatiscertainlythecase
withlsr.Listingthetenmostrecentlymodifiedfilesinadirectory
isaseasyasthis:
ls-lt$HOME/dld|head
Togetthetenoldestisonlyonelettermore:
ls-lrt$HOME/dld|head
Tocutthatdowntothefivenewestfilesaddsmorecharacters:
ls-lt$HOME/dld|head-5
Nowthat’slongenoughtojustifyascript,isn’tit?
Usage
lsr[OPTIONS][FILE...]
Bydefault,lsrliststhetenmostrecentlymodifiedfiles.The-n
optionallowstheusertoselectanynumberoffiles:
$lsr-n3/usr/local/bin/*
-r-xr-xr-x1chrischris1444Nov2
17:41/usr/local/bin/sym2file
-rwxr-xr-x1chrischris4158Nov2
12:36/usr/local/bin/standard-funcs
-r-xr-xr-x1chrischris1926Nov2
12:29/usr/local/bin/rand-funcs
The-ooptiontellsthescripttoshowtheoldestfiles:
$lsr-on3/usr/bin
-rwxr-xr-x1rootroot741588Dec12
2000plain-gmc
-rwxr-xr-x1rootroot750432Dec12
2000gmc
-rwxr-xr-x1rootroot1939Jan10
2003newsgroups
The-aoptionincludesdotfiles:
$lsr-an3
drwx------4chrischris8192Nov2
23:34mail
-rw-------1chrischris44035Nov2
23:33.jnewsrc-nin.as
drwxr-xr-x3chrischris16384Nov2
23:31scripts
Whenlistingfilesatthecommandline,Ialwayswantalong
listing,butImaywantjustthenameswhenpipingtheresultto
anotherprogram.Theshortoption,-s,givesmethat:
$lsr-asn4
scripts
.
.followup
.bash_history
TheScript
num=10##Defaulttotenmostrecent
files
short=0##Uselonglistingby
default
opts=an:o:s
do
case$optin
a)ls_opts="$ls_opts-a";;
n)num=$OPTARG;;
o)ls_opts="$ls_opts-r";;
s)short=1;;
esac
done
shift$(($OPTIND-1))
##Add-ttolsoptionsand-lifnot
usingshortmode
case$shortin
1)ls_opts="$ls_opts-t";;
*)ls_opts="$ls_opts-l-t";;
esac
ls$ls_opts"$@"|{
##Ifthefirstlineis"total...",itis
removed
read-rline
case$linein
total*);;
*)printf"%s\n""$line";;
esac
cat
}|head-$num
Summary
Thescriptsinthischapterrunthegamutfromtotallyinteractive
(undup)tofullyautomated(dfcmp).Somewhereinbetween,symfix,
sym2file,andzrmcandotheirworkunhindered,orcanpromptthe
userforconfirmationofeachaction.Thoughdesignedforuseatthe
commandline,lsrneedsnointeraction,andcanbeusedinscripts.
Thisistypicalofafilemanagementsystem,ofwhichthese
scriptswouldbeonlyasmallpart.Sometaskscanbeautomated,
butyouwillalwayswanttocontrolsomethingsyourself.In
additiontothese,andthefile-agingscriptsinChapter14,youwill
probablyneedagoodfilemanager.SomepeopleswearbyMidnight
Commander(mc),othersbyKonqueror.Myfavoriteisgentoo,
whichwasbasedontheAmigafilemanager,DirectoryOpus.Many
applicationsalsohaveinteractivefilemanagers:lynx,emacs,pine,
andothers.
Inaddition,youwillprobablyneedtowritesomemorescripts
todealwithyourownparticularsituation.
Chapter10:Screenplay:Thescreen–
funcsLibrary
(1) Ontario,Canada
DataEntryonterminalscreensinvolvedpositioningthecursoron
thescreenatvariousplaces,displayingpiecesofinformationin
boldorreversetext.Theusualmethodofpositioningthecursorand
settingattributessuchasboldandreverseinvolvestput,aUnix
utilitythatoutputsscreencodesfortheparticularterminalor
terminalemulationyouareusing.Evenwithprocessorsrunningat
multigigahertzspeeds,callinganexternalcommandmultipletimes
canproduceanoticeabledelay.
Foracertainproject,Christooktheeasywayoutandplacedthe
cursor-positioningcodesforterminalsinaprintatfunction,andthe
printingattributes(bold,reverse,underline)invariables.He
rationalizedthisbyassuringhimselfthat,asthiswasaspecialized
script,itwouldneverberunanywhereelse.
Naturally,ayearlaterhisclientreplacedthesystem,andthe
newterminalsdidnotunderstandthesamecodes.Sohesetoutto
replacetheoldcodeswiththenew,andittookalmostnotimeatall.
Theoldcodeswereinfunctionsinasinglefile;thechangeoverwas
lessthananhour’swork.(Actually,therewereafewrenegadelines
ofcodeinotherfilesthattookalittlelongertoferretout,butthat
wasaminorproblem).Thisisaperfectexampleofhowchanges
thatcanbreakyourcode.
Becausetherearemanydifferenttypesofterminal(thetermcap
filethatdefinesthecodescancontainentriesforhundredsof
terminals),tputisoftenusedtoproducethecorrectscreencodesfor
positioningthecursorandsettingorremovingbold,underline,or
otherattributes.Onterminalsthatsupportcolor,tputcangenerate
thecodesforsettingthebackgroundandforegroundcolors.
Unfortunately,tputhasproblems(besidesthetimeittakesasan
externalcommand).First,somesystemsdonothaveit.Theseare
rarenowadays,butnotunknown.Second,twodifferentsetsof
commandsareacceptedbytput.Mostversionsonlyacceptoneset
ortheother.Bothofthefollowingcommandsplacethecursoratthe
fifthcolumnonthethirdrow:
tputcup24
tputcm42
ThefirstcommandworksonaGNU/Linuxsystem,thesecond
onacomputerrunningFreeBSD.Fortputcommandstowork
correctlyonallsystems,youwouldhavetotestwhichversionwas
present,andusedifferentcommandsdependingontheresult.Then
therearethosesystemswithoutanyversionoftput.
Thevastmajorityofterminalsandterminalemulationsusethe
AmericanNationalStandardsInstitute(ANSI)setofcodes.These
werefirstimplementedontheDECvt100terminalin1978,andare
oftenreferredtoasvt100codes.Communicationsprogramssuchas
hyperterminalandputtyusethesecodes,orhaveoptionstouse
them,inwhatisoftencalledvt100emulation.Thefunctionsinthis
chapterallusetheANSIstandardcodes,whichwillworkinany
terminalhavingvt100emulation.
Thescreen-funcslibraryfirstsourcesstandard-funcsfrom
Chapter1andthescreen-varsfile:
.
standard-funcs
.
screen-vars
10.1screen-vars—VariablesforScreen
Manipulation
AfterplayingaroundwithANSIscreencodesoverthelast2-3
decades,onemightnotstillrememberthemall.Manyofusdon’t
evenrememberalltheonesusedfrequently.Luckily,the
computer’smemoryisbetter,andyoucanletitdothework.
HowItWorks
Agoodwaywastouseconstantsorvariableswithunderstandable
names.Thesecouldbeplacedtogetherinonefiletobeusedontop
ofeveryscriptthatmightneedthem.
Usage
Thesevariablesareusedmostlyinthefunctionsdefinedinscreenfuncs,buttheycanbeuseddirectlyinascriptoratthecommand
line(aftersourcingscreen-vars).Forexample,toclearthescreen
withoutusinganexternalcommand,thereis$CLS:
printf"$CLS"
Toprintabaracrossthescreen,plugthereverseattributeinto
$set_attr:
printf
"\n$set_attr%${COLUMNS}.${COLUMNS}s\n"$reverse
Todrawthesamebarwithwordsinitisjustamatterofprinting
acarriagereturn(sothatthecursorstaysonthesameline)andthen
thetext:
{
printf"\n$set_attr%${COLUMNS}.${COLUMNS}s\r"$reverse
printf"***%s***\n""StarsoftheANSIscreen"
}
The$cu_row_colvariable,like$set_attrandothers,canbe
usedastheformatstringinaprintfstatementtoplacethecursor
anywhereonthescreen;forexample,thetopleftcorner:
printf"$cu_row_col"11
Foranythingmorecomplex(andevenforpositioningthe
cursor),Itisrecommendusingthefunctions,ratherthanthe
variables.Thatway,whenyourunintoanonstandardterminal,all
youneedchangearethefunctions(theformatforthevariables
mightnotbethesame).
TheScript
.standard-vars##loadstandard
variables($NL,$ESC,etc.fromChapter
1
)
##attributes:printf"$set_attr"$bold
bold=1
dim=2##maynotbesupported
underline=4
blink=5##maynotbesupported
reverse=7
hidden=8##maynotbesupported
##screencodes
CSI=$ESC[##Control
SequenceIntroducer
NA=${CSI}0m##Resetattributes
tothedefault
CLS=${CSI}H${CSI}2J##Clearthe
screen
cle=$ESC[K##Cleartoendof
line
##cursormovement
cu_row_col="${CSI}%d;%dH"##position
cursorbyrowandcolumn
cu_up=${CSI}%dA##mvcursorupNlines
cu_down=${CSI}%dB##mvcursordownNlines
cu_d1=${CSI}1B##mvcursordown1line
cu_right=${CSI}%dC##mvcursorrightNcolumns
cu_left=${CSI}%dD##mvcursorleftNcolumns
cu_save=${ESC}7##savecursorposition
cu_restore=${ESC}8##restore
cursortolastsavedposition
cu_vis="${CSI}?12l${CSI}?25h"##visiblecursor
cu_invis="${CSI}?25l"##invisible
cursor
cu_NL=$cu_restore$cu_d1$cu_save##movetonext
line
##setattributes
set_attr=${CSI}%sm##setprinting
attribute
set_bold=${CSI}1m##equiv:printf
"$set_attr"$bold
set_ul=${CSI}4m##equiv:printf"$set_attr"
$underline
set_rev=${CSI}7m##equiv:printf
"$set_attr"$reverse
##unsetattributes
unset_bold=${CSI}22m
unset_ul=${CSI}24m
unset_rev=${CSI}27m
##colors(precedeby3forforeground,
4forbackground)
black=0
red=1
green=2
yellow=3
blue=4
magenta=5
cyan=6
white=7
fg=3
bg=4
##setcolors
set_bg="${CSI}4%dm"##e.g.:printf"$set_bg"$red
set_fg="${CSI}3%dm"##e.g.:printf"$set_fg"
$yellow
set_fgbg="${CSI}3%d;4%dm"##e.g.:
printf"$set_fgbg"$yellow$red
10.2set_attr—SetScreen-PrintingAttributes
Whendealingwithacharacter-basedterminal,thewaysofchanging
textappearancearelimited.TheANSImodesforterminalsare
bright,dim,underscore,blink,reverse,andhidden.Ifyouarelucky,
yourterminalwillnotsupporttheblinkattribute(“Wedon’tneed
noblinkin’text!”),butyouwillwantboldandreverse.
HowItWorks
The$set_attrvariablecontainsaformatstringthatisusedasa
formatstringforprintf.
Usage
set_attr[ATTRIBUTE[;ATTRIBUTE]]
Theblink,dim,andhiddenattributesmaynotbeimplemented
onyourterminal.Theyareincludedinscreen-varsfor
completeness.Idonotrecommendusingthem.
Multipleattributescanbespecifiedinasinglecall(notethe0to
turnoffanyattributescurrentlyineffectbeforesettingnewvalues):
set_attr0$bold$underline
Colorsmayalsobesetwithset_attr,buttherearefunctionsfor
thatwhichareeasiertouse(seeset_fg,set_bg,andset_fgbglater
inthischapter).
TheScript
set_attr()
{
for_attrin"${@:-0}"
do
case$_attrin
##ifargumentisnegative,removetheattribute
-"$bold")printf"$set_attr""22";;##21
maynotwork
-[0-9]*)printf"$set_attr""2${_attr#-}";;
##positivenumber;addanattribute
[0-9]*|"")printf"$set_attr""${1:-0}";;
esac
done
}
10.3set_fg,set_bg,set_fgbg—SetColorsfor
PrintingtotheScreen
Thevastmajorityofusersnowadayswillberunningtheshellina
colorenvironment,whetheritistheLinuxconsoleorantelnet/ssh
window.Thisallowsmoreattractivedisplaysfromshellscripts.To
easetheuseofcolor,thesefunctionssettheforegroundand
backgroundcolors.
HowItWorks
Definedinscreen-varsatthebeginningofthischapter,theprintf
formatstringsset_fg,set_bg,andset_fgbgarethebasisforthese
threefunctions.
Usage
set_fg[0-7]##setthe
foreground(printing)color
set_bg[0-7]##setthe
backgroundcolor
set_fgbg[0-7][0-7]##setthe
foregroundandbackgroundcolors
Thoughtheargumentsareallasingledigitfrom0to7,thereis
noneedtorememberwhichnumberrepresentswhichcolor;that’s
alreadydoneforyouwiththecolordefinitionsinscreen-vars.You
canjustusethecolorvariables:
set_fg$red
set_bg$white
set_fgbg$blue$yellow
Thesesettingsaffectsubsequenttext,notcolorsalready
displayed.Anargumentof-1inanypositionsetsallcolorsand
attributestothedefault.Anyotherargumentreturnsanerror.
Thesefunctionsmaybeusedanywhereset_attrisusedinthis
orfuturechapters.
TheScript
##settheforegroundcolor
set_fg()
{
case$1in
[0-7])printf"$set_fg""$1";;
-1|"")printf"$NA";;
*)return5;;
esac
}
##setthebackgroundcolor
set_bg()
{
case$1in
[0-7])printf"$set_bg""$1";;
-1|"")printf"$NA";;
*)return5;;
esac
}
##setbothforegroundandbackground
colors
set_fgbg()
{
case$1$2in
[0-7][0-7])printf"$set_fgbg""$1""$2";;
*-1*|"")printf"$NA";;
*)return5;;
esac
}
10.4cls—CleartheScreen
Whenusingthefullscreen(orwindow)forscriptoutput,one
usuallystartswithacleanslate.Theclearcommandwilldothis,
butitisanexternalcommand;isthereawaytodoitjustusingthe
shell?
HowItWorks
Sinceclearprintsyourterminal’scodeforclearingthescreen(or
enoughblanklinestoclearitifthereisnosuchcode),itsoutputcan
becapturedinavariableandprintedtothescreenwhenneeded.
TheANSIcodetoclearthescreenissetinscreen-vars,butitcan
bedeleted,andtheclsfunctionwillusethecorrectcodeforyour
terminal,settingthevariableatthesametime.
Usage
cls
Therearenoargumentsnoroptionstothissimplefunction.
TheScript
If$CLSisnotdefined,thisfunctionwillusetheoutputofclear,
assigningitto$CLSaswellasprintingtheresult.Normally,$CLS
willhavebeendefined;clearisafall-back(thatwillbecalledno
morethanonce)incasescreen-varshasnotbeensourced.
cls()
{
printf"${CLS:=$(clear)}"
}
Notes
Oneevening,whileplayinginapubtrivialeague,itoccurredto
Christhattheclsfunctioncouldbeenhanced.(Don’taskwhyit
cametohimduringasetofquestionsonmedievalhistory!)Hewas
inthemiddleofwritingthischapteratthetime,sohejotteditdown
onascrapofpaper.Itaddsthecapabilityofchangingthecolorsand
otherattributesofthescreenaswellasclearingit.
Itusesthreefunctionsthataredefinedearlierinthischapter:
set_attr,set_bg,andset_fgbg.Whencalledwithoutany
arguments,itsbehaviorisexactlythesameasthesimplerversion.
Whenthereisoneargument,thatvalueisusedtosetthe
backgroundcolor.Withtwoarguments,thefirstdefinesthe
foregroundcolor,thesecondthebackgroundcolor.Withmorethan
twoarguments,allremainingvaluesareusedtosetattributessuch
asbold,underline,andreverse.Thefirsttwoargumentsmustbe
digitsintherange0to7,representingtheeightcolorsavailablein
theANSIsystem.
cls(){
case$#in
0);;
1)set_bg$1;;
2)set_fgbg$2$1;;
*)set_fgbg$2$1
shift2
set_attr"$@"
;;
esac
printf"${CLS:=$(clear)}"
}
10.5printat—PositionCursorbyRowand
Column
Youcanusetwomethodstosendtheoutputofascripttoaterminal
screen.Thetraditionalmethodusestheterminalasifitwerea
printerorteletypewriter(whichistheoriginoftheabbreviationtty
forthescreenorterminal).Inthismode,aseachlineisprinted,the
paper,orscreenimage,isscrolledup.Oldlinesfalltothefloor,or
disappearoffthetopofthescreen.It’ssimple,anditismorethan
adequateformanyapplications.
Thesecondmethodisthesubjectofmostofthischapter;it
treatsthescreenasablackboardorcanvas,andprintstospecific
pointsonitssurface.Itoverprintspreviouslywrittensections,using
spacestoerasewhennecesssary.Itmayprinttextincolumnsorat
specificlocationsonthescreen.Theterminalbecomesarandomaccess,ratherthanserial,device.Acommandtoplacethecursorat
anychosenpointonthescreenisneededtoimplementthismethod.
HowItWorks
Mostterminalsandterminalemulationsuseanescapesequenceto
positionthecursor.ThestandardANSIsequenceisESC[<ROW>;
<COL>m.Thisstringhasbeenconvertedtotheprintfformatstring
cu_row_col(seescreen-varsearlierinthischapter).Theprintat
functionusesitscommand-lineargumentsandthisvariabletoplace
thecursorattherequestedcoordinates.
Usage
printat[row[column[string]]]
Ifeithertheroworcolumnargumentismissingorisanempty
string,a1isused,representingthetoprowortheleftmostcolumn.
Ifstringispresent,itisprintedatthespecifiedlocationwithouta
terminatinglinefeed.(Inthismodeofscreenprinting,linefeedsare
neverused,astheycouldcausethescreentoscroll,ruiningits
formatting.)
Besidesusingprintatdirectly,itsoutputcanbestoredina
variableandusedmanytimes,savingoncomputationaltime.For
example,todisplayinformationonerecordatatimefrom
/etc/passwd,usethisscript:
.screen-funcs##loadvariablesand
functions
form=$(##storeemptyformina
variable
printat21"UserName:"
printat230"UID:"
printat240"GID:"
printat31"Fullname:"
printat41"HomeDirectory:"
printat51"Shell:"
)
print_form()##print$formwithfields
filledin
{
cls
set_attr0##turnoffbold,reverse,etc.
printf"%s""$form"
set_attr$bold
printat217"$id"
printat235"$uid"
printat245"$gid"
printat317"$name"
printat417"$home"
printat517"$shell"
}
##loopthroughthelinesin
/etc/passwd
whileIFS=:read-ridxuidgidname
homeshell
do
print_form
set_attr$reverse
printat121"<PRESSANYKEYTOCONTINUE>$CR"
##standardinputisredirected,soreadkeyboardfrom
/dev/tty
get_keyx</dev/tty
case$xinq)break;;esac
done</etc/passwd
printf"$cle"
set_attr0
Figure10-1showstheresultingoutput.
Figure10-1. Onerecordfromthepasswordfile
TheScript
printat()
{
printf"$cu_row_col"${1:-1}${2:-1}
##removethefirsttwoarguments
case${2+X}in
X)shift2;;
esac
##printanyremainingarguments
case"$*"in
*?*)printf"%s""$*";;
esac
}
Notes
IfnotusinganANSIterminal(theVT100emulationcommonto
mostterminalprogramsunderstandstheANSIcodes),youneedto
knowthecorrectcommandforyourterminal.Ifyoursystemhas
tput,oneoftputcupYX,tputcmYX,ortputcmXY,will
positionthecursoratcolumnXonrowY(countingfrom0,0asthe
topleftcorner;printatandtheterminalcodescountfrom1,1).
YourTERMvariablewilldeterminethestringrequired.Ifyoudon’t
havetput,youmustlookupthecorrectstringinyourterminal’s
manualortermcaporterminfofile.
WhilesearchingtheWebforwaystoaccommodateallterminals
onallsystems,Icameacrossthissnippet.It’sfromascriptthat
implementsashellversionofthecurseslibrary(usedforscreen
manipulationinCprograms).Itpipestheoutputoftputthroughsed
tocreateacursorpositioningcommandcustomizedforwhatever
terminalyouareusing.
evalCMD_MOVE=\ècho\"`tputcup`\"\|sed
\\\
-e\"s/%p1%d/\\\\\${1}/g\"\\\
-e\"s/%p2%d/\\\\\${2}/g\"\\\
-e\"s/%p1%02d/\\\\\${1}/g\"\\\
-e\"s/%p2%02d/\\\\\${2}/g\"\\\
-e\"s/%p1%03d/\\\\\${1}/g\"\\\
-e\"s/%p2%03d/\\\\\${2}/g\"\\\
-e\"s/%p1%03d/\\\\\${1}/g\"\\\
-e
\"s/%d\\\;%dH/\\\\\${1}\\\;\\\\\${2}H/g\"\\\
-e\"s/%p1%c/'\\\\\\\ècho\\\\\\\${1}P|
dc\\\\\\\`'/g\"\\\
-e\"s/%p2%c/'\\\\\\\ècho\\\\\\\${2}P|
dc\\\\\\\`'/g\"\\\
-e\"s/%p1%\'\'%+%c/'\\\\\\\ècho
\\\\\\\${1}32+P|dc\\\\\\\`'/g\"\\\
-e\"s/%p2%\'\'%+%c/'\\\\\\\ècho
\\\\\\\${2}32+P|dc\\\\\\\`'/g\"\\\
-e\"s/%p1%\'@\'%+%c/'\\\\\\\ècho
\\\\\\\${1}100+P|dc\\\\\\\`'/g\"\\\
-e\"s/%p2%\'@\'%+%c/'\\\\\\\ècho
\\\\\\\${2}100+P|dc\\\\\\\`'/g\"\\\
-e\"s/%i//g\;s/%n//g\"\`
WhenruninastandardANSIterminal,thisproducesthe
equivalentofthe$cu_row_colvariableIintroducedinscreen-vars.
Butthephrase“molassesinJanuary”leapedintomyheadwhenI
realizedwhatitproducedforaWyse60terminal:acommand
substitutionstringthatmakestwocallstothe(external)dc
commandtodothearithmeticeverytimeitisused.Icouldmake
thissnippetconsiderablyfaster(inshellswithabuilt-inprintf)by
usingthechrfunctionfromChapter3,butitstillwouldn’tworkon
systemsthatusecminsteadofcupasthetputattribute.They
generateanerror,notaformatstring,whencmhasnoarguments.
Hereisaversionforapopularterminal,theWyse60:
printat()
{
printf"\e="
chr-n$((32+${1:-1}))$((32+${2:-1}))
case${2+X}in
X)shift2;;
esac
case"$*"in
*?*)printf"%s""$*";;
esac
}
10.6put_block_at—PrintLinesinaColumn
AnywhereontheScreen
Youwanttoprintblocksofinformationatdifferentlocationsonthe
screen;forexample,alistatcolumn65onrow1andsubsequent
rows,andanotheratcolumn10onrow9andbelow.Youcould
prefernottohavetocallprintatforeachline.
HowItWorks
Threevariablesdefinedinscreen-vars,$cu_save,$cu_d1and
$cu_restorearecombinedin$cu_NL,whichrestoresapreviously
savedcursorposition,movesthecursordownoneline,andsaves
thecursorposition.When$cu_NLisusedattheendofaprintf
formatstringinsteadof$NL,eachargumentisprintedimmediately
belowthepreviousone,insteadofa—tthebeginningofthenext
line.
Usage
put_blockARG1...
put_block_atROWCOLARG1...
Theargumentstoput_blockareprinted,onetoaline,aligned
beneaththefirstcharacterofthefirstline.
$cls;printat125;put_blockThe
quickbrownfox
The
quick
brown
fox
Thesecondfunction,put_block_at,incorporatesprintat,so
thiswillgivethesameoutputasthepreviouscommandline:
$cls;put_block_at125Thequick
brownfox
Hereisanexampleofbuildingamorecomplexscreenusing
printatandput_block_at.First,listsarestoredinthreevariables:
{
q=$(cut-d:-f1/etc/passwd)
w=$(cut-d:-f7/etc/passwd|sort-u)
e=$(grep-v'^#'/etc/shells)
Thenwesourcethescreen-funcslibrary(whichloadsscreenvars),turnoffthecursor,clearthescreen,andprintthelistsat
differentplaces:
.screen-funcs-sh
cls
bar=============================================
printf"$cu_invis"
put_block_at165$q
put_block_at910$w
put_block_at930$e
printat340"Listofusers==>"
printat81
printf"%63.63s""$bar$bar$bar"
printat710"Shellsused"
printat730"Shellsallowed"
sleep5
printat$LINES1"$cu_vis"
}
Theresultlookssomethinglikethis:
List
ofusers==>bin
ShellsusedShellsallowedman
==============================================================
lp
/bin/bash/bin/ash
/bin/false/bin/bash
/bin/sh/bin/csh
/bin/sync/bin/sh
/usr/bin/es
data
/usr/bin/ksh
/bin/ksh
/usr/bin/rc
/usr/bin/tcsh
/bin/tcsh
/usr/bin/zsh
/bin/sash
/bin/zsh
/usr/bin/esh
/bin/dash
/usr/bin/screen
TheScript—
put_block()
{
printf"$cu_save"##savecursorlocation
printf"%s$cu_NL""$@"
}
put_block_at()
{
printat$1$2"$cu_save"
shift2
printf"%s$cu_NL""$@"
}
10.7get_size—SetLINESandCOLUMNS
Variables
Todrawtothescreen,weneedtoknowhowbigitis.Thebashshell
automaticallyinitializesthevariables$LINESand$COLUMNS,but
othershellsarenotsocooperative.
HowItWorks
Bothsttyandtputcanprintthesizeofthescreen,butIhavefound
sttysizetobemoreconsistentacrossdifferentsystemsthantput
linesandtputcols;sttysizeisusuallyfaster,thoughnotenough
toswaythedecision.NeithercommandisrequiredbythePOSIX
specificationstogivethedesiredinformation,butall
implementationsI’veseendo.
Usage
get_size
If,forsomereason,sttysizedoesnotproducetheexpected
output,defaultsof80columnsand24linesareused.Thesearethe
traditionalvaluesforacharacterterminal.
TheScript
get_size()
{
set--$(sttysize2>/dev/null)
COLUMNS=${2:-80}
LINES=${1:-24}
exportCOLUMNSLINES
}
10.8max_length—FindtheLengthofthe
LongestArgument
Theoutputoftheput_blockfunctionsmaycollidewithand
overwritetextalreadyonthescreen,makingitillegible.For
example,ifIrunthesetwocommands:
put_block_at110JonathonChristopherEmilio
put_block_at110SueJohnBob
I’llendupwithnonsenseonthescreen:
Sueathon
Johnstopher
Boblio
Theprint_blockfunctionthatfollowsneedstoknowthelength
ofthelongestiteminthelistsoitcaneraseanytextalreadyonthe
screen.
HowItWorks
Unfortunately,thereisnochoicebuttocyclethroughalltheitems,
keepingtrackofthelongest.POSIXshellparameterexpansionis
usedtoobtainthenumberofcharactersineachargument.
Usage
_max_length[ARG...]##storeresult
in$_MAX_LENGTH
max_length[ARG...]##printresult
Aswithmanyotherfunctionsinthisbook,max_lengthispaired
withanunderscorefunction,_max_length,forefficiency.Ifno
argumentsaregiven,_MAX_LENGTHwillcontain0;noerroris
generated.
TheScript
_max_length()
{
##initialize_MAX_LENGTHwiththelengthofthe
firstargument
_MAX_LENGTH=${#1}
[$#-eq0]&&return##ifnoarguments,just
return
shift##removethefirstargument
forvar##cyclethroughtheremainingargs
do
["${#var}"-gt"$_MAX_LENGTH"]&&
_MAX_LENGTH=${#var}
done
}
max_length()
{
_max_length"$@"
printf"%s\n"$_MAX_LENGTH
}
10.9print_block_at—PrintaBlockofLines
AnywhereontheScreen
Onanemptyscreen,put_blockisanadequatemethodofprintinga
list.Whenthescreenisfull,theoutputmaybejumbledwith
existingtext.Tomakereadingeasier,Iwouldliketoeraseallthe
charactersinarectangletocontainthelist.
HowItWorks
Afterfindingthelengthofthelongestiteminthelistbyusing
_max_length,awidthspecificationisusedwithprintftopadthe
lines.Alineisaddedbeforeandafterthelist,andaspaceisprinted
beforeandaftereachpaddeditem.
Usage
print_blockARG1...
print_block_atROWCOLARG1...
Theusageofprint_blockisidenticaltothatofput_block,but
theresultisquitedifferentiftextisalreadyonthescreen.
.string-funcs;.screenfuncs##loadlibraries
cls##clearthescreen
_repeat'BBBBBBB'
${COLUMNS}0##buildstring10lineslong
printf"%s"
"$_REPEAT"##printlongstring
sleep
1##takeanap
box=$(print_blockThequickbrownfox)
##storeblockinvariable
printat266
"$box"##printnearrightofscreen
set_attr
$bold##turnonboldprinting
printat316
"$box"##printnearleftofscreen
Withallthenastydetailshiddenawayinthefunctionlibrary,
thisshortscriptproducesthis:
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
TheBBBBBBBB
BBBBBBBBBBBBBBB
TheBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBquick
BBBBBBBB
BBBBBBBBBBBBBBB
quick
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBbrownBBBBBBBB
BBBBBBBBBBBBBBB
brown
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBfoxBBBBBBBB
BBBBBBBBBBBBBBB
foxBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
TheScript
print_block_at()
{
printat$1$2"$cu_save"
##_BLOCK_PARMSstoresvaluesforclearingthearea
later
_BLOCK_PARMS="$1$2"
shift2
print_block"$@"
}
print_block()
{
_max_length"$@"
[$_MAX_LENGTH-gt$(($COLUMNS-2))]&&
pb_w=$(($COLUMNS-2))||
pb_w=$_MAX_LENGTH
##printfre-usesitsformatstringuntil
##alltheargumentsareprinted
printf"%-${pb_w#-}.${pb_w#-}s$cu_NL""""$@"""
##storeparamtersforusewiththeclear_area_at
function
_BLOCK_PARMS="$_BLOCK_PARMS$(($_MAX_LENGTH+2))
$(($#+2))"
}
10.10vbar,hbar—PrintaVerticalorHorizontal
Bar
Thegraphicalpossibilitieswithshellscriptsarelimited,but
horizontalandverticalbarsarewellwithinitscapabilities.What’s
neededarefunctionstomakedrawingthemeasy.
HowItWorks
Withprintf’swidthspecification,anynumberofcharactersofa
stringcanbeprinted.Withaloopandthe$cu_NLvariabletorestore
thecursorpositionandmovetothesamecolumninthenextline,a
barcanberepeatedasmanytimesasnecessary.Thebarcanbethe
widthofasinglecharacteroraswideasthescreen.
Usage
vbarHEIGHT[WIDTH[STRING]]
hbar[WIDTH[STRING]]
Bydefault,bothcommandswilldrawinvisiblebars.Tomake
themvisible,eitheranatttributemustbeset(reverse,oracolor
differentfromthebackground),orastringmustbespecified.The
defaultwidthisonecharacter.
Toprintaverticalbardownthemiddleofthescreen:
cls##clearthe
screen
set_attr$reverse##setreverse
printing
vbar$(($COLUMNS/2))##printbar
Toprintahorizontalbaracrossthemiddleofthescreen:
cls
printat$(($LINES/2))1##position
cursor
set_attr$reverse##set
reverseprinting
hbar$COLUMNS##print
bar
Toprintaverticalredbar(onlyonacolorterminal,ofcourse)
10characterswideand20linesdeep:
set_attr$bg$red##setbackgroundred
vbar2010##drawthebar
TheScript
vbar()
{
printf"$cu_save"##savecursorposition
##Ifastringisgiven,repeatituntilitislong
enoughto
##fillthewidthofthebar
case$3in
?*)vb_char=$3
while[${#vb_char}-lt${2:-1}]
do
vb_char=$vb_char$vb_char$vb_char
done
;;
*)vb_char="";;
esac
##Aloopstartswithvbar_setto0,andincrements
ituntil
##thespecifiednumberoflineshavebeenprinted.
vbar_=0
while[$vbar_-lt${1:-$LINES}]
do
printf"%${2:-1}.${2:-1}s$cu_NL""$vb_char"
vbar_=$(($vbar_+1))
done
}
##Ahorizontalbarisaverticalbar
onelinedeep.
hbar()
{
vbar1${1:-$COLUMNS}"$2"
}
10.11center—CenteraStringonNColumns
Atitlesometimeslooksbetterwhencenteredonthescreenorina
box.Thecalculationsneededtodothepositioningarenot
complicated,andareeasilywithinthecapabilitiesofthePOSIX
shell.
HowItWorks
Usingparameterexpansiontogetthelengthofthestringtobe
printed,thisisaddedtothewidthinwhichitistobecentered,and
thetotalishalved.Thenprintfisusedtopositionthestringflush
rightacrossthatmanycharacters.
Usage
center[-NUM]
string
WhenNUMisnotsupplied,centerprintsitsnon-option
argumentsacrossthewidthofthescreen,asdefinedbythe$COLUMNS
variable.Nolinefeedisprinted.
centerThislineiscenteredacrossthe
entirescreen;printf"\n"
centerAndthisiscenteredbeneath
it;printf"\n\n"
center-40Centeredin40
columns;printf"\n"
center-40Alsoin40
columns;printf"\n\n"
center-60Centeredin60
columns;printf"\n"
center-60Onceagainin60
Thisscriptproduces:
Thislineiscenteredacrossthe
entirescreen
Andthisiscentered
beneathit
Centeredin40columns
Alsoin40columns
Centeredin60columns
Onceagainin60columns
Ifthestringexceedstheallottedspace,itwillbetruncated:
${center-30Nowisthe
time;printf"\n"
center-30forallgoodmentocometo
theaidof;printf"\n"
center-30the
party;printf"\n"
}
Nowisthetime
forallgoodmentocometotheai
theparty
TheScript
center(){
case$1in
-[0-9]*)c_cols=${1#-}
shift
;;
*)c_cols=${COLUMNS:-78};;
esac
string="$*"
c_len=$((($c_cols-${#string})/2+
${#string}))
printf"%${c_len}.${c_len}s""$*"
}
10.12flush_right—AlignStringwiththeRight
Margin
Thoughnotusedasoftenascenter,Idosometimeswanttoprint
textagainsttherightmarginofthescreenoracolumn.
HowItWorks
Whenawidthissuppliedtoaformatspecificationinaprintf
formatstring,bydefaultitprintsitsargumentalignedattherighthandsideofthewidth.Thismakestheflush_rightfunctionasnap
towrite.
Usage
flush_right[-NUM]string
Aswithcenter,ifNUMisnotsupplied,theentirewidthofthe
screenisusedasthecanvas.
flush_rightThislineisflushright
acrosstheentirescreen;printf"\n"
flush_rightAttherightofthe
screen;printf"\n\n"
flush_right-40Flushrightacross40
columns;printf"\n"
flush_right-40Thesame
again;printf"\n\n"
flush_right-60Flushrightacross60
columns;printf"\n"
flush_right-60Oncemorewith60
Ifanystringhadbeenlongerthanitsallottedspace,itwould
havebeentruncated,buttheseallfit:
Thislineisflushrightacrossthe
entirescreen
Attherightofthescreen
Flushrightacross40columns
Thesameagain
Flushrightacross60columns
Oncemorewith60columns
TheScript
flush_right(){
case$1in
-[0-9]*)fr_cols=${1#-}
shift
;;
*)fr_cols=${COLUMNS:-78};;
esac
printf"%${fr_cols}.${fr_cols}s""$*"
}
10.13ruler—DrawaRulerAcrosstheWidth
andHeightoftheWindow
Whendesigningascreenorcheckingtheoutputofascript,itoften
helpstohaveareferenceforthenumberingofrowsandcolumns.
HowItWorks
Thehorizontalrulerusesprintftotruncatethenumberingatthe
edgeofthescreen,andalooptocountthelinesdowntheside.
Usage
ruler
Thisisasimplefunctionwithnooptionsandnoarguments.The
outputisshowninFigure10-2.
Figure10-2. Therulerfunction’soutputdisplayedinawindow
TheScript
ruler()
{
printat11
##buildtensandunitsstringstoprintacrossthe
topofthescreen
tens="1234
tens="$tens890"
tens=$tens$tens$tens
printf"%${COLUMNS:=80}.${COLUMNS:=80}s\n""$tens"
vruler
printat21
one2ten=1234567890
while[${#one2ten}-lt$COLUMNS]
do
one2ten=$one2ten$one2ten$one2ten
done
printf"%-${COLUMNS}.${COLUMNS}s\n""$one2ten"
}
vruler()
{
n=$((${bar_line:-1}))
printat11"$cu_save"
while[$n-le$LINES]
do
printf"%2s$cu_NL"$n
n=$(($n+1))
done
}
10.14box_block,box_block_at—PrintText
SurroundedbyaBox
Todrawattentiontoablockoftext,itsometimeshelpstodrawa
boxaroundit.Iwanttobeabletocontrolthecolorsoftheboxand
thebackgroundandtextcolorsinsideit.
HowItWorks
Twovariablesareusedtosettheattributes,includingcolors,forthe
outlineandtheinterior.Atechniquesimilartothatusedin
print_blockisusedtoprintthebox.
Usage
bx_border=$(...)bx_body=$(...)
box_block_atCOLROW[STRING...]
box_block[STRING...]
Thetwovariables,bx_borderandbx_body,mustbesettostrings
thatcontrolthecolorsandattributesfortheboxanditsinterior.
Theseareusuallytheoutputofset_attrandset_fgbg.Usually,
set_attr0shouldbeusedfirsttoclearanyexistingsettings.For
example,thissetstheboxtoblueandprintsthetextinboldredona
yellowbackground:
bx_border=$(set_attr0;set_bg$blue)
bx_body=$(set_attr0$bold;set_fgbg
$red$yellow)
box_block_at33Troopingthecolors
Figure10-3showstheoutput.
Figure10-3. Aboxedmessageprintedusingbox_block_at
Thelocationandsizeoftheresultingboxisstoredin
$_BLOCK_PARMS,whichcanbeusedastheargumentto
clear_block_at,whichispresentedinthenextsectionofthis
chapter.
TheScript
box_block_at()
{
printat$1$2
_BLOCK_PARMS="$1$2"
shift2
box_block"$@"
}
box_block()
{
_max_length"$@"
[$_MAX_LENGTH-gt$(($COLUMNS-4))]&&
pb_w=$(($COLUMNS-4))||pb_w=$_MAX_LENGTH
##usethemaximumlengthoftheargumentsforwidth
specification
pb_w1=$pb_w.$pb_w
pb_w=$(($pb_w+2)).$(($pb_w+2))
##printthetoplineusingthe$bx_borderattributes
printf"$cu_save$bx_border"
printf"%-${pb_w}s$cu_NL"""
##printfre-usesitsformatstringtillallthe
argumentsareprinted
printf"$bx_border$bx_body%-${pb_w}s$bx_border
$cu_NL""""$@"""
printf"$bx_border%-${pb_w}s$bx_body$cu_NL"""
##storeparametersforusewithclear_area_at
_BLOCK_PARMS="$_BLOCK_PARMS$(($_MAX_LENGTH+4))$((
$#+4))"
}
10.15clear_area,clear_area_at—ClearanArea
oftheScreen
Whenusingthescreenasablackboard,Ifrequentlyneedtoerasean
areatoreusefornewordifferentinformation.Ineedtobeableto
specifythelocationandsizeofthearea,andsetthebackground
color.
HowItWorks
Asinseveralotherfunctionsinthischapter,thebackgroundcoloris
setbeforethefunctioniscalled.Thefunctionloopsthedesired
numberoftimes,printingspacestothewidthspecified.
Usage
clear_areaWIDTHHEIGHT
clear_area_atROWCOLWIDTHHEIGHT
Theprint_block_atandbox_block_atfunctionsstoretheir
parametersin$_BLOCK_PARMS.Thisvariablecanbeusedasthe
argumenttotheclear_area_atfunction.
cls
set_fgbg$green$white
print_block_at233"Divinginto
Python"\
"PracticalSubversion""ShellScripting
Recipes"
sleep2
set_attr
clear_area_at$_BLOCK_PARMS
Ifyoumakeanothercalltoprint_block_atorbox_block_at
beforeyoucallclear_area_at,youwillneedtosavethevaluesin
$_BLOCK_PARMStoanothervariable,becauseitwillbeover-written:
print_block_at22331234Pleaseclean
theboard
block=$_BLOCK_PARMS
sleep2##dootherstuff
clear_area_at$block
TheScript
clear_area()
{
printf"$cu_save"##savecursor
position
n=0##setcounter
while[$n-le$2]##looptill
counter...
do##reachesthe
valueof$2
printf"%${1}.${1}s$cu_NL"""##printspacesto
WIDTH
n=$(($n+1))##increment
counter
done
}
clear_area_at()
{
printat$1$2##positioncursor
clear_area$3$4##clearthearea
}
10.16box_area,box_area_at—DrawaBox
AroundanArea
SometimesIwanttoclearanareaonthescreenandputabox
aroundit,clearinganareaofthescreensothatIcanplace
informationinitatalatertime,perhapsinrepsonsetouserinputor
thecompletionofanothertask.Figure10-4showstheresult.
Figure10-4. Anareaclearedandboxed
HowItWorks
Aswithsomeoftheothertasksinthischapter,therearetwo
functionsfordrawingaboxaroundanarbitraryarea:onethat
performsataskatthecurrentcursorposition,andanotherthat
movestoacertainpositionbeforedoingitsthing.Unlikethe
previousfunctions,theseusedifferenttechniques.
Sincebox_area_atknowsitslocation(thefirsttwoarguments),
itcanreturntothatpointtodrawtheinnerbox.Ontheotherhand,
box_areadoesn’thavethatinformation,andtosaveandrestorethe
positionwouldbemorecomplicated.Itthereforeproceedsonaonewayjourneyfromtoptobottom.
Usage
bx_border=$(...)bx_body=$(...)
box_areaWIDTHHEIGHT
box_area_atROWCOLWIDTHHEIGHT
Aswithbox_block,thebox_areafunctionstaketheircolorsand
attributesfromthebx_borderandbx_bodyvariables.
cls
bx_border=$(set_attr0;set_bg$yellow)
$blue$magenta)
box_area_at331212
TheScript
box_area_at()
{
##settheborderstyle
printf"$bx_border"
##clearthewholeareawiththebordercolor
clear_area_at$1$2$3$(($4-1))
##returntothetopleftcorner,
##onesquaredownandonetotheright
printat$(($1+1))$(($2+1))
##settheinteriorstyle
printf"$bx_body"
##clearasmallerareawiththebodycolor
clear_area$(($3-2))$(($4-3))
}
box_area()
{
printf"$bx_border$cu_save"
##printthetoplinewiththebordervalues
printf"%${1}.${1}s$cu_NL"""
##createastringtocontaintheborderforeach
line
ba_border="$bx_border$bx_body"
##calculatethewidthoftheinsidearea
ba_w=$(($1-2))
##loopuntilwereachthebottomline
ba_=2
while[$ba_-lt$2]
do
##printtheborderleftandrightoneachline
printf
"$ba_border%${ba_w}.${ba_w}s$ba_border$cu_NL"
ba_=$(($ba_+1))
done
##printthebottomlineofthebox
printf"$bx_border$cu_save"
printf"%${1}.${1}s$cu_NL"""
}
10.17screen-demo—SavingandRedisplaying
AreasoftheScreen
Ausefultechniqueistosavetheresultsinvariablessothatboxes
canberedrawnoverlaterones.Thisdemonstrationscriptprints
threeboxeswithtext,andthenmovesthembackandforthinfront
ofeachother:
HowItWorks
Theoutputofthreecallstobox_block_atarestoredinvariables.
Thisincludesthecursor-positioningcodes,sotheblocksappearat
thesamelocationonthescreenwhentheyareprinted.SeeFigure
10-5fortheresult.
Figure10-5. Thescreen-demoscriptatwork
Usage
screen-demo
TheScript
screen-funcs##loadthe
libraries
cls$cyan$cyan##clearthe
screen(withalternateversionofcls)
printf"$cu_invis"##hidethecursor
##setattributesandcolorsforbox1
bx_border=$(set_attr0;set_bg$yellow)
$white$red)
##printbox1toavariable
box1=$(box_block_at510"AMidsummer
Night’sDream"\
"WilliamShakespeare")
bx_border=$(set_attr0;set_bg$black)
bx_body=$(set_attr0$reverse$bold;
set_fgbg$red$white)
—
box2=$(box_block_at318"SilasMarner"
"GeorgeEliot")
bx_border=$(set_attr0;set_bg$blue)
bx_body=$(set_attr0;set_fgbg$black
$white)
box3=$(box_block_at728"Howthe
GrinchStoleChristmas""Dr.Seuss")
##printallthreeoverlappingboxes
printf"$box1"
printf"$box2"
printf"$box3
##takeanap
sleep1
##moveboxestothefront
printf"$box1"
sleep1
printf"$box3"
sleep1
printf"$box2"
sleep1
printf"$box1"
sleep1
printf"$box3"
#restorecursorandmovedownafew
lines
printf"$cu_vis\n\n\n\n"
Summary
Unlikemostfunctionsinthisbook,thosepresentedinthischapter
havelittleapplicationatthecommandline;theyaredesigned
almostexclusivelyforuseinotherscripts.Theyareusefulin
interactivescriptssuchasgamesordataentryforms.Theycanbe
usedforpresentations(putaseriesof“slides”inabasharray,and
movethroughthem,forwardorbackward,withakeypress)or
menus.TheyarenotasfancyasGUIapplications,butcanbeused
tocreateimpressivedisplaysinverylittletime.
Chapter11:Aging,Archiving,and
DeletingFiles
(1) Ontario,Canada
Filesgrow.It’safactoflife.Therearelogfilesin/var/log(they
maybein/var/adm,orsomewhereelse,dependingonyoursystem).
InmymaildirectoryIhavesaved-messages,andsent-mail,and
filesforthevariousmailinglistsI’mon.InmyChessdirectory,
fics.pgnstoreseverygameofchessIplayonline.FilesinmyNews
directoryswellwithmyUsenetposts.
MostmodernUnixsystemsofferalogrotationprogramthatcan
beconfiguredtosuitmanyofyourneeds.Filescanberotateddaily,
weekly,monthly,orwhenevertheyreachacertainsize.Theycanbe
compressedornot,asyouwish.Traditionally,logrotationfiles
havebeennumbered,withthemostrecentalwayshavingasuffixof
.0,thenextoldesthaving.1,andsoon.Whenanewrotationwas
performed,thefilesallhadtoberenamed.Theonewiththehighest
numberwouldbedeleted.
Mye-mailclient,bydefault,renamessaved-messagesandsentmailfilesatthebeginningofeverymonth.Itusesfilenameswitha
three-letterabbreviationforthemonth.Mye-mailclientsortsthem
chronologically,butthestandardUnixutilitiesdonot.
Ipreferfileagingtorotation,byrenamingthefilewitha
datestampintheinternationalformat;messageswouldbecome
messages_2004-11-27ormessages_2004-11-27_23.12.39,orany
otherdateformatIchoose.Filesagedonceamonthonlyneedthe
yearandthemonth,fics_2004-09.Withthismethod,thefilesnever
needfurtherrenaming.Asystemneedstobeinstalledtomoveor
deletefilesastheygetold,butthat’ssimpler,andmoreflexible,
thanhavingtorenameeveryfileintherotationeverytime.Thiscan
bedoneeitherbyremovingfilesaftertheyreachacertainage,orby
keepingonlythenewestfiles;scriptsforbothmethodsarepresented
afterthescripttocreatedate-stampedfiles.
11.1date-file—AddaDatestamptoaFilename
Iwantcertaine-mailfilesmovedatthebeginningofeachmonth,
withtheyearandmonthappendedtothename.MyUsenetposts,
whichareseparatedbynewsgroup,shouldbemovedwhentheyget
toobig(forme,toobigis100,000bytes,foryouitmaybe10
kilobytesor1gigabyte;itneedstobeconfigurable).Iwantmy
onlinechessfilerenamedeverymonth.Logfilesneedtobe
renamedeverymonthoreveryweek,andsomeeveryday.
HowItWorks
Thesimplestsolutionofallwasformychessfile,fics.pgn.The
scriptthatloadsxboard(agraphicalchessboardthatconnectsto
http://www.freechess.orgorothersites)specifiesthefileinwhich
gamesaretobesaved.Wecanmodifythatscripttoincludethedate
inthefilenamewithfics_`date+%Y-%m`.pgninsteadoffics.pgn.
Problemsolved.Wouldn’titbeniceifallprogramscreatedtheirlog
filesthatway?
Well,itturnsoutthatmye-mailclient(pine)nowhasanoption
touseaYYYY-MMformatforitsprunedfiles(itwasn’ttherethe
lasttimeIlooked!).PerhapsIdon’tneedthisscriptafterall?No
suchluck;therearetoomanyotherplaceswhereIdo.
Myoriginalplanforthisscriptwasshortandsweet:
mv"$file""${file}_$(date+%Y-%m-%d)"
Ifwewantedtobeabletochangethedateformat,wecanadd
anoption.Weknowthatmyfile_2004-11-26.cwouldlookbetter
thanmyfile.c_2004-11-26,sowecanaddanoptiontoputthedate
beforetheextension.Thescriptgrewuntilithadallthefeatures
necessary(andafewthatjustfeltgood)forpruninganyfilethat
growsoveraperiodoftime.
Usage
date-file[OPTIONS]FILE...
Bydefault,date-fileaddsadateintheformat.YYYY-MM-DDto
theendofeachfilenamesuppliedonthecommandline.Ifthe
filenameis-,filenamesarereadfromthestandardinput.The
optionscanchangeeverythingfromtheformatofthedatetothe
locationofthefile’sfinalrestingplace.
-c:Createanew,emptyfile(becausesomeloggingutilities
writetoalogfileonlywhenitalreadyexists).
-dDIR:MovefiletodirectoryDIR.Ifthedirectorydoesn’t
exist,date-filewilltrytocreateit.Ifitisunsuccessful,itwill
exitwithanerror.
-D:Includetimeanddateinthenewfilenamebyusingthe
formatstring_%Y-%m-%d_%H.%M.%S.
-fFMT:UseFMTasthedateformatstringinsteadof.%Y-%m%d.
-hCMD:ExecutethecommandCMDaftermovingthefile.Ifa
programkeepsitslogfileopen,itcouldcontinuetowriteto
theoldfile.SendingtheHUPsignalcausesittoreloadits
configurationfilesandcloseanyfilesithasopen.Onmy
Mandrakelinuxsystem,forexample,logrotateuses
/usr/bin/killall-HUPsyslogdtotellsyslogdtoclosetheold
logfileandusethenewone.
-s:Insertthedatebeforethefinalsuffix.
-SSUFF:Insertthedatebeforethesuffix,SUFF,ifitmatchesthe
endofthefilename.
-v:Verbosemode;printtheoldandnewnamesforeachfile.
-z:Compressthefilewithgzip.
-ZCMD:CompressthefilewithCMD,(bzip2,forexample).
Hereareafewsamples,firstwiththedefaultdateformatand
usinggzip-vtocompresstheresultingfile:
$date-file-Z'gzip-v'yule.log
yule.log_2004-11-26:80.0%--replaced
withyule.log_2004-11-26.gz
Withthedateformatstring,_%a,theabbreviateddayoftheweek
isusedinsteadofthefulldate,the-coptiontellsdatefiletocreate
anotherfileofthesamename,and-vputsitinverbosemodesowe
canseetheresults:
$date-file-c-v-f_%aeaster.ogg
"easter.ogg"->
"easter.ogg_Fri"
With-s,thedateisinsertedbeforethe.gifsuffix,and-dlogs
movesthefileintothelogsdirectory:
$date-file-vsdlogsxmas.gif
"xmas.gif"->
"logs/xmas_2004-12-25.gif"
TheScript
##Thechangeofnameisdonebythe
mv_filefunction.
##USAGE:mv_filePATH/TO/FILE
mv_file()
{
##ifthefilein$1doesnotexist,returnanerror
[-f"$1"]||return5
if[$pre_suffix-eq0]
then
##Bydefault,date-fileaddsthedatetotheend
ofthename
newname=$1$DATE
else
##Ifpre_suffixhasbeensettoanythingother
than0,
##thedateisinsertedbeforethesuffix
basename=${1%$suffix}##getfilename
withoutsuffix
postfix=${1#$basename}##getthesuffix
alone
newname=$basename$DATE$postfix##placethedate
betweenthem
fi
##Ifadestinationdirectoryhasbeengiven,the
newnameis
##thedestinationdirectoryfollowedbyaslashand
the
##filenamewithoutanyoldpath
[-n"$dest"]&&newname=$dest/${newname##*/}
##Ifinverbosemode,printtheoldandnew
filenames
if[$verbose-ge1]
then
printf"\"%-25s->\"%s\"\n""$1\"""$newname"
>&2
fi
##Movethefiletoitsnewnameorlocation
mv"$1""$newname"
##Iftheoptionisselected,createanew,empty
file
##inplaceoftheoldone
[$create-eq1]&&touch"$1"
##Iftheapplicationthatwritestothelogfile
keepsthe
##fileopen,senditasignalwiththescriptin
$hup.
[-n"$hup"]&&eval"$hup"
##Compressifselected
[-n"$gzip"]&&$gzip"$newname"
return0
}
##Allthesedefaultsmaybeoverridden
bycommand-lineoptions
pre_suffix=0##donotplaceDATE
beforesuffix
suffix=".*"##usedifpre_suffix
issetto1
dfmt=_%Y-%m-%d##defaultdateformat:
_YYYY-MM-DD
verbose=0##verbosemodeoff
dest=##destination
directory
create=0##donotre-create
(empty)file
gzip=##donotuse
compression
hup=##commandtointerrupt
programwritingtofile
opts=cd:Df:h:o:sS:vzZ:
do
case$optin
c)create=1;;##Createnew,
emptyfile
d)[-d"$OPTARG"]||##Movefiles
todifferentdir
mkdir"$OPTARG"||##createif
necessary
exit5##return
errorifnotpossible
dest=${OPTARG%/};;##remove
trailingslash
D)dfmt=_%Y-%m-%d_%H.%M.%S;;##usetimeas
wellasdate
f)dfmt=$OPTARG;;##usesupplied
dateformat
h)hup=$OPTARG;;##commandto
runaftermv
s)pre_suffix=1;;##placedate
beforesuffix
S)suffix=$OPTARG##placedate
beforespecified
pre_suffix=1;;##suffix
v)verbose=$(($verbose+1));;##verbosemode
z)gzip=gzip;;##compress
filewithgzip
Z)gzip=$OPTARG;;##compress
filewithsuppliedcmd
*)exit5;;##exiton
invalidoption
esac
done
shift$(($OPTIND-1))##Remove
optionsfromcommandline
##Storethedatein$DATE
eval$(date"+DATE=$dfmt")
##Loopthroughfilesonthecommand
line
forfile
do
case$filein
-)##Ifa$fileis"-",readfilenamesfrom
standardinput
whileIFS=read-rfile
do
mv_file"$file"
done
;;
*)mv_file"$file";;
esac
done
Notes
Whenagingafileonthefirstofthemonth,normallyinthesmall
hoursofthemorning,wewantthedatetoreflecttheprevious
month,whichiswhenitwasgenerated.Wecouldconsideraddinga
dateoffsetoption.ThatwouldbeeasywiththeGNUor*BSD
versionofdate,butcomplicatedwithagenericdatecommand.
Wecanagethemonthlyfilesshortlyaftermidnightonthefirst
ofeachmonth,sowegeneratethedatewithatimezonethatisone
hourearlier.Chrisisintheeasterntimezone,whichisfivehours
behindCoordinatedUniversalTime(UTC).BysettingtheTZ
variabletoanearliertimezone(Central),Wecanuseyesterday’s
(i.e.,lastmonth’s)date:
TZ=CST6CDTdate-fileFILENAME
11.2rmold—RemoveOldFiles
Manyofushaveseveraldirectorieswherefilescollectuntilwecan
getaroundtocleaningthemout.Theyaremostlytemporaryfiles,
downloads,andagedfilescreatedwiththedate-filescript.
Downloadedfilesareusuallytarballsorotherpackagesthatarenot
neededaftertheircontentshavebeenextracted.Vastareasofmy
harddrivesfillupwitholdandunneededfiles.Whenthedrivesfill
up,weneedtodosomethingaboutthem.Weneedascripttoclear
outoldfiles.
HowItWorks
Beingtheconsummatepackrat,IhatetogetridofanythingunlessI
amsureIwillneverneeditagainunderanycircumstancesno
matterhowunlikely,andthenIhavetothinktwice.Theonlyway
thesefileswilleverbecleanedout(shortofthediskbeingfull)isif
Imakeiteasytoremoveoldfiles,orelseautomatetheprocess.The
rmoldscriptcanremovetheoldfileswithouthumanintervention.
Thescriptreadsthelistofdirectoriestobeprunedfromafile
specifiedonthecommandlineor,bydefault,from
$HOME/.config/rmold.dirs.Thedefaultistoremovefilesthatare
morethan30daysold,buteachdirectorycanhavealongeror
shorterperiodassignedtoit,andthedefaultcanbechangedbya
command-lineoption.
Usage
rmold[OPTIONS]
Thelistofdirectoriestobeprunedisplacedin
$HOME/.config/rmold.dirs;thisfilelocationcanbechangedwith
the-coption.(Severalscriptsinthisbookkeeptheirconfiguration
filesin$HOME/.config,creatingitifnecessary.Ifthedirectorydoes
notexist,goaheadandmkdirit.)Thefilecontainsalistof
directories,onetoaline,withanoptionalage(indays)followinga
space,comma,ortab.
Thedefaultageforfiles(whennotspecifiedinthermold.dirs
file)issetwith-d.Ifyouarenotsureaboutdeletingyourfiles,theioptionputsrmintointeractivemode,anditwillaskforyour
approvalbeforeremovinganyfiles:
$rmold-i
rm:removeregularfile
`/home/chris/tmp/sh-utils-2.0/man/chroot.1'?y
`/home/chris/tmp/sh-utils-2.0/man/date.1'?y
`/home/chris/tmp/sh-utils-2.0/man/dirname.1'?n
With-mDEST,adestinationdirectorycanbegiven,andthefiles
willbemovedtothatdirectoryratherthanbedeleted.Thefinal
option,-v,worksonlywiththeGNUversionsofrmandmv,as
othersdonothaveaverbosemode:
$rmold-v
removed`/home/chris/tmp/sh-utils2.0/man/echo.1'
removed`/home/chris/tmp/sh-utils2.0/man/env.1'
removed`/home/chris/tmp/sh-utils2.0/man/false.1'
Ifanemptyfilenameisgivenwiththe-coption,thescriptwill
readthedirectoriesfromthestandardinput.Ifthatisaterminal,the
userwillbepromptedtoenteradirectoryandage:
$rmold-vc''
Nameofdirectory:
/tmp/xx
Deletefilesolderthan(days):22
removed`/home/chris/tmp/xx/zzqw31055'
removed`/home/chris/tmp/xx/zzer21252'
removed`/home/chris/tmp/xx/zzty27584'
Ifstdinisnotaterminal,theinputwillbeexpectedinthesame
formatasthermold.dirsfile(directorynamewithoptionalagein
daysseparatedbyaspace,tab,orcomma):
$echo
/tmp/xx33|rmold-sh-vc''
removed`/home/chris/tmp/xx/zzqw22088'
removed`/home/chris/tmp/xx/zzer4828'
removed`/home/chris/tmp/xx/zzty26562'
TheScript
##Defaultvaluesthatcanbechanged
withcommand-lineoptions
days=30##
ageindays
dir_file=$HOME/.config/rmold.dirs##
filecontaininglistofdirs
rm_opts=##
optionstorm
opts=c:d:im:v
do
case$optin
c)dir_file=$OPTARG;;
d)days=$OPTARG;;
i)rm_opts="$rm_opts-i";;
m)move=1##movefilesinsteadof
deleteingthem
dest=$OPTARG##directorytoplacefilesin
##create$destifitdoesn’texist,
##andexitwithanerrorifitcannotbe
created
[-d"$dest"]||mkdir"$dest"||exit5
;;
v)rm_opts="$rm_opts-v";;
esac
done
shift$(($OPTIND-1))
##if$dir_filecontainsafilename,
checkthatitexists,
case$dir_filein
-|"")dir_file=;;
*)[-f"$dir_file"]||exit5;;
esac
##if$dir_fileexists,useit
if[-n"$dir_file"]
then
cat"$dir_file"
##otherwise,ifstdinisaterminal,
promptuserfordirectory
elif[-t0]
then
printf"Nameofdirectory:">&2
readdir
eval"dir=$dir"
[-d"$dir"]||exit5
printf"Deletefilesolderthan(days):">&2
readd
printf"%s%d\n""$dir""$d"
else##stdinisnotaterminal,so
passinputthroughunchanged
cat
fi|
whileIFS=','readdirdays_x
do
case$dirin
"")break;;##skipblanklines
\#*);;##skipcomments
*)##lastaccesstime(-atime)isusedrather
thanlast
##modificationtime
[$move-eq0]&&
find"$dir"-atime+${days_:-$days}-typef
\
-execrm$rm_opts{}\;
</dev/tty||
find"$dir"-atime+${days_:-$days}-typef
\
-execmv$rm_opts{}"$dest"
\;</dev/tty
;;
esac
done
11.3keepnewest—RemoveAllbuttheNewest
orOldestFiles
Inthealt.linuxnewsgroup,BernhardKastneraskedhowtodelete
allthefilesinadirectoryexceptforthefourmostrecently
modified.Kastneransweredthequestionhimselfalittlelater,after
readingthesedmanual:
rm`ls-t*|sed-e1,4d`
Forthefileagingsystem,wehavethermoldscript,which
deletesfilesbasedonage.Forsomedirectories,orsomefilesin
somedirectories,keepingaspecificnumberworksbetter.Tofitinto
myschemeofthings,thescriptwouldhavetobeabletomove
ratherthandeletefiles,andbeabletokeepaquantityotherthan
fourfiles.
HowItWorks
BybreakingKastner’sscriptintotwoparts,andusingvariables
insteadofhard-codingthenumber,flexibilityisadded.Thefirst
partoftheresultingscriptbuildsalistoffilenames,thesecondpart
operatesonthem.Command-lineoptionsareusedtoadjustthe
numberoffilesandtheirdisposition.Asabonus,thereisanoption
tokeeptheoldestratherthanthenewestfiles.
Usage
keepnewest[OPTIONS][FILE...]
Withoutanyoptionsorotherarguments,keepnewestdeletesall
butthefourmostrecentfilesinthecurrentdirectory.Ifalistoffiles
isspecified,usuallydoneintheformofapatternsuchassentmail*,allfilesmatchingthepattern,exceptthefourmostrecent,
willbedeleted.Tochangethenumberoffileskept,usethe-n
option:
keepnewest-n13
Tomovethefilesratherthandeletethem,specifyadestination
directorywith-d:
keepnewest-dold-files
Tomovethefilesandcompressthemwithgzip,use-cand
specifyadirectory:
keepnewest-cdold-files
Adifferentcompressioncommandmaybegivenasthe
argumentto-C:
keepnewest-dold-files-Cbzip2
The-ooptionkeepstheoldestratherthanthenewestfiles.To
keepthe12oldestfilesbeginningwithaandmovealltheothers
(thatbeginwitha)tothe$HOME/a-olddirectoryandcompressthem
withgzip,usethis:
keepnewest-d
TheScript
/a-old-n12-o-ca*
##defaultvalues
n=4##numberoffilestokeep
dest=##ifadirectoryis
specified,movethefiles
compress=##compressfilesthat
havebeenmoved
old=##ifsetto-r,keep
oldestfiles
##Usually,NL(newline)isdefinedby
sourcingstandard-vars,
##butsinceonlyonevariableis
needed,Ihaveputthedefinition
##in-line
NL='
'
opts=cC:d:n:o
do
case$optin
c)compress=gzip;;##usethedefaultcommand,
gzip
C)compress=${OPTARG:-bzip2}
##Ifthecommand(withoutoptions)doesn’t
exist,
##exitwitherrorcode6
type${compress%%*}>/dev/null||exit6
;;
d)dest=$OPTARG
[-d"$dest"]||##checkthatthedirectory
exists
mkdir"$dest"||##ifitdoesn’t,createit
exit5##ifthatfails,exit
withanerror
;;
n)case$OPTARGin
##exitwitherrorifOPTARGcontains
##anythingotherthanapositive
integer
*[!0-9]*)exit4;;
esac
n=$OPTARG
;;
o)old=-r;;##keepoldestfiles,notnewest
esac
done
shift$(($OPTIND-1))
##Weassumethatanerrorhasbeen
madeifnofilesaretobe
##kept,sinceascriptisnot
necessaryforthat(rmisenough)
##Ifyouwanttheoptiontoremoveall
files,commentoutthe
##followingline.
[$n-eq0]&&exit5
##Storethelistoffilesina
variable.
filelist=`ls$old-t"$@"|sed-e
"1,${n}d"`
##Bysettingtheinternalfield
separatortoanewline,spacesin
##$filelistwillbetreatedaspartof
thefilename,notasa
##spacebetweendiffrerentnames
##Note:thisscriptwillfailifany
filenamescontainanewline
##character(theyshouldn’t).
IFS=$NL
case$destin
"")##nodestinationdirectoryisgiven,sodeletethe
files
rm$filelist;;
*)##movefilestoanotherdirectory
mv$filelist"$dest"
##if$compressisdefined,cdtothedestination
directory
##andcompresseachfile
[-z"$compress"]||(
cd"$dest"||exit3
forfilein$filelist
do
$compress${file##*/}
done
)||exit
;;
esac
Summary
Thescriptsinthischapterarethebuildingblocksofafile-aging
system.Whilethesecommandscanbeusedmanually,theirvalue
increaseswhentheyarecombinedinascripttailoredtoyourneeds.
Chapter12:CoveringAllYour
Databases
(1) Ontario,Canada
Adatabaseissimplyacollectionofinformation.Theinformationis
usuallydividedintorecordsthatmaybesimpleorcomplex.There
maybeahandfulofrecords,orafewdozen.Ortheremaybe
thousandsormillionsofrecords.
Eachrecordinadatabasemaybeasingleword,oritmay
compriseanumberoffields.Thesefieldsmaybeself-contained
data,ortheymaybekeystoanotherdatabase.Arecordmaybeone
ormorelinesinafile,oranentirefilebyitself.
The/etc/passwdfileonaUnixsystemisadatabasecontaining
onerecordforeachuser.Eachrecordcontainssevenfields,
separatedbycolons.Atypicalentryforrootis
root:x:0:0:root:/root:/bin/sh
Theword-finderandanagramscriptsinChapter4usea
databaseofmorethanahundredthousandwords,eachwordbeinga
record,andaCompoundsfilewiththreetab-separatedfieldsoneach
line(aphrasewithspacesremoved,theoriginalphrase,andthe
wordlengths).
InChapter17,thegeneralledger(GL)entryscript,gle,stores
transactionsinafilethatispartofarelationaldatabase;someofthe
fieldscontainkeysthatcanbelookedupinothertables.Aclient
maybeenteredintheGLassun;thiskeycanbelookedup(using
thelookupscript,forexample)inatableofclientstogetthefull
nameofthecompany,theaddress,phonenumber,oranyother
informationitcontains.Transactionsarechargedtoaccounts
representedbyabbreviationsthatcanbelookedupinthechartof
accounts;ARcanbelookeduptorevealAccountsReceivable.
Evenamembershipdatabasewiththousandsofrecords,suchas
theoneChriswrotefortheTorontoFree-Net,workswellwithall
programswritteninshellscripts.Asinglerecordfromthedatabase
lookslikethis(butallononeline,notfour):
bq833:bBFRa3et3r0zM:43833:25:Fred
Arnold:/homes/d02/33/bq833:
/usr/local/FreePort/bin/BBmenu:691DonBowie
Avenue:Apt.1:Toronto:Ontario:
Canada::1949-09-29:2005-10-09:416-5557940:[email protected]:416-555-7490::
English:Lawless::416-555-9407:20:1995-0714:2005-01-20_17_16_24
TheonscreenoutputisshowninFigure12-1.
Figure12-1. TheTorontoFree-Netmembershipdatabasescreen
Forcomplicated,interrelateddatabaseswhereentrymaybe
donebymorethanoneperson,andwheretransactionsmustpassthe
ACID1test,afulldatabasemanagementsystemshouldbeused
(PostgreSQLisaprominentopensourceexample).Themajorityor
personaldatabasesdonotrequireanythingclosetothatcomplexity,
andcanbemaintainedwithafewshellscripts.
Thescriptsinthischapterworkwithsingle-linerecords.Some
useadelimitertoseparatefields(oftenataboracomma);others
useafree-formrecordwithoutanydefinedfields.
12.1lookup—FindtheCorrespondingValuefor
aKey
Inthegeneralledgersystem,abbreviationsareusedtoreferto
clients,suppliers,andaccounttypes.Thefullinformationforeach
clientisarecordinanotherfile.Youwouldneedacommandto
lookuptheabbreviationandretrievetheclient’sfullname.The
clientfilecouldlooklikethis(thoughitactuallycontainsmore
fields—andclients),withfieldsseparatedbytildes:
sun
TheTorontoSun 333KingStreet
East Toronto Ontario 416-555-1234
gt
GoodTimes 25SheppardAvenueWest North
York Ontario 416-555-2468
Sincewewillbeusingtheresultinascript,weneedthefield
extractedfromtherecord.
HowItWorks
Iftheabbreviationisalwaysinthefirstfield,andthefullnameis
alwaysinthesecondfield,averysimplescriptisallyouneed(this
oneassumesatab-delimitedformat;the$TABvariablecontainsatab
character,somethingweusuallysetbysourcingstandard-vars
fromChapter1):
lookup(){grep"^$1$TAB""$2"|cut-f2;
}
Note
ifyouhavejumpedchaptersoraretryingoutthiscommand
directly,simplysetthevariableTABtotheseparator,thetildein
thisexample.
Thiscanbecalledwiththeabbreviationtolookupandthenameof
thefileitisin:
$lookupsun$HOME/clients
TheTorontoSun
Theflexibilityofthescriptcanbeenhancedbyusingawkwith
command-lineoptionstospecifyfieldsforthekeyorthevaluetobe
retrieved,andtosetthedelimiter.
Usage
lookup[OPTIONS]KEYFILE
Therearethreeoptions,allofwhichtakeanargument:
-f:Setsthefielddelimiter
-k:Setsthenumberofthefieldwhichcontainsthelook-upkey
-v:Specifieswhichfieldtoreturn
Ifyouwantedtolookupauser’shomedirectory,thesixthfield
inthecolon-delimitedfile,/etc/passwd,youcoulddothis:
$lookup-f:-v6chris/etc/passwd
/home/chris
Whenweuselookupinascript,wehavethenamesofthefields
asvariables;forexample,whenworkingwith/etc/passwd,make
theseassignments:
_user=1
_passwd=2
_uid=3
_gid=4
_fullname=5
_homedir=6
_shell=7
Then,tofindauser’sfullname,thevariablecanbeusedinstead
ofthenumber:
$lookup-v$_fullname-f:chris
/etc/passwd
ChrisF.A.Johnson
Sincelookupisusedprimarilyinscripts,theresultislikelytobe
placedinavariable.Forexample,thiswillfindmichael’sshell:
shell=$(lookup-v$_shell-f:michael
/etc/passwd)
Tofindoutwherethefirstpartitionofyoursecondharddriveis
mounted,thedefaultvalues(thekeyinthefirstfieldandthevalue
inthesecond)canbeusedtogettheinformationfrom/etc/fstab:
$lookup/dev/hdb1/etc/fstab
/usr
TheScript
##defaultfieldnumbersmaybechanged
bycommand-lineoptions
key=1##fieldtouseasthekey
val=2##fieldcontainingdesired
value
F=##usedefaultfieldseparator
whilegetoptsk:f:v:opt
do
case$optin
f)F=$OPTARG;;
k)key=$OPTARG;;
v)val=$OPTARG;;
esac
done
shift$(($OPTIND-1))
str=$1##stringtolookup
shift##removelook-upstring
##if$Fisempty,thefieldseparator
isnotchanged
awk${F:+-F"$F"}'$key==str{print
$val}
'key=$keyval=$valstr="$str""$@"
shdb-funcs:ShellDatabaseFunctionLibrary
Whenyouneedadatabase,suchasforbookkeepingormembership
records,wecouldwriteitfromscratch.Usually,theirrequirements
aredifferentsowecanreuseverylittlecode.Thereare,however,a
numberoffunctionsthatmanycouldfindusefulwhendealingwith
databases.Thesefunctionsarecollectedinshdb-funcsandare
designedforflexibility.Theymakeuseofshellarraysandother
featuresofbash.
Thefirsttaskistoloadthestandard-funcslibraryfromChapter
1,whichinturnloadsstandard-vars.
.standard-funcs
12.2load_db—ImportDatabaseintoShellArray
Movingbackandforthamongtherecordsinadatabaseismade
mucheasierbyhavingthemallinmemory.
HowItWorks
Bysettingtheinternalfieldseparator,IFS,toanewline,theentire
databasefilecanbereadintomemoryinasinglestatement,and
eachlineassignedtooneelementinanarray.Thelocalkeyword
limitsthescopeofthevariabletothefunctionanditschildren.
Usage
load_dbFILE
There’snothingcomplicatedaboutthisfunction,andifit
weren’tnecessarytoresetIFS,wecouldhaveusedin-linecode
ratherthanaseparatefunction.Theentiredatabaseisstoredinthe
${db[]}array.
TheScript
load_db()
{
localIFS=$NL##SetIFSlocallytoanewline
localshdb=$1##Thefirstpositionalparameter
isthefile
db=(`<"$shdb"`)##Slurp!
}
Notes
Ifyouhaveadatabasewithlinescommentedout,removethose
linesfromthefilebeforeslurpingitintothearray.Thisversionof
load_dbwilleliminateanylinethatbeginswith$NB(bydefault,#),
ignoringanyleadingspacesortabs:
load_db()
{
localIFS=$NL##SetIFSlocallytoanewline
localshdb=$1##Thefirstpositionalparameter
isthefile
localNB=${NB:-#}##NBcontainsthecomment
character
db=(`grep-v"^[$TAB]*$NB""$shdb"`)##Slurp!
}
12.3split_record—SplitaRecordintoFields
ManyUnixconfigurationfilesare,infact,databases.Inmostcases,
theyhaveonerecordtoaline.Thefieldsmaybedelimitedbyone
ormorewhitespacecharacters(e.g.,/etc/fstab),acolon(e.g.,
/etc/passwd,asshowninFigure12-2),orsomeothercharacter.The
filesystemcanbeviewedasahierarchicaldatabasewithfields
separatedbyslashes.
Figure12-2. Thefirst22recordsinmy/etc/passwd
HowItWorks
BysettingtheIFSvariabletothedelimiter,therecordissplitand
storedinashellarray.
Usage
split_recordRECORD
Bydefault,fieldsareseparatedbyoneormorewhitespace
characters,spaceandtab.Thiscanbechangedbysettingthe
variableDELIM.Inthepasswordfile,thedelimiterisacolon.Here
theentryforrootisextractedfrom/etc/passwdwithgrepandsplit
intoanarray:
$DELIM=:split_record$(grep^root:
/etc/passwd)
$printf"%s\n""${record_vals[@]}"
root
x
0
0
root
/root
/bin/bash
Thefieldsin/etc/fstabareseparatedbywhitespacecharacters.
Here,sedpullsthefirstlinefromthefile,andthatisusedasthe
argumenttosplit_record:
$split_record$(sed-n1p/etc/fstab)
$printf"%s\n""${record_vals[@]}"
/dev/hda1
/
ext3
defaults
1
1
TheScript
split_record()
{
localIFS=${DELIM:-$TAB}##ifDELIMnotset,uses
spaceandTAB
localopts=$-##saveshelloptionflags
set-f##disablepathname
globbing
record_vals=($*)##storeargumentsin
array
##resetglobbingonlyifitwasoriginallyset
case$optsin
*f*);;
*)set+f;;
esac
}
12.4csv_split—ExtractFieldsfromCSV
Records
Onecommondatainterchangeformatusescomma-separatedvalues
(CSVs).MostspreadsheetprogramscanimportandexportCSV
files.Mycreditunionsallowsmetodownloadmyaccounthistory
inaCSVformat.Wecandownloadstockquotesfrom
finance.yahoo.comorcurrencyexchangeratesfromtheEuropean
CentralBankinCSVformat.
Therearevariationsintheformat,butbasically,thefieldsare
separatedbycommas,andnon-numericfieldsareenclosedin
quotationmarks.Inthisway,textfieldscancontaincommasor
quotationmarks.Splittingtheserecordsintotheirfieldsismore
complicatedthanwithasimplecharacter-delimitedrecord.
HowItWorks
Ratherthansplittingtherecordinasinglestep,asinsplit_record,
itisnecessarytoloopthroughtherecordonefieldatatime,
removingquoteswhentheyencloseafield.
Usage
csv_split"RECORD"
TheRECORDmustbeasingleargument:
$q='123,"Steinbeck,John","TheGrapes
ofWrath",1939,328'
$csv_split"$q"
$pr1"${record_vals[@]}"
123
Steinbeck,John
TheGrapesofWrath
1939
328
Textfieldsmaycontainquotationmarksandcommas,but
shouldnotcontainadoublequotefollowedbyacomma.
TheScript
csv_split(){
csv_vnum=0##fieldnumber
csv_record=${1%"${CR}"}##removecarriagereturn,
ifany
unsetrecord_vals##weneedapristine
(global)array
##removeeachfieldfromtherecordandstorein
record_vals[]
##whenalltherecordsarestored,$csv_recordwill
beempty
while[-n"$csv_record"]
do
case$csv_recordin
##if$csv_recordstartswithaquotationmark,
##extractupto'",'orendofrecord
\"*)csv_right=${csv_record#*\",}
csv_value=${csv_record%%\",*}
record_vals[$csv_vnum]=${csv_value#\"}
;;
##otherwiseextracttothenextcomma
*)record_vals[$csv_vnum]=${csv_record%%,*}
csv_right=${csv_record#*,}
;;
esac
csv_record=${csv_right}##theremainsofthe
record
##Ifwhatremainsisthesameastherecordbefore
theprevious
##fieldwasextracted,itisthelastfield,so
storeitandexit
##theloop
if["$csv_record"="$csv_last"]
then
csv_record=${csv_record#\"}
record_vals[$csv_vnum]=${csv_record%\"}
break
fi
csv_last=$csv_record
csv_vnum=$(($csv_vnum+1))
done
}
12.5put_record—AssembleaDatabaseRecord
fromanArray
Oncearecordhasbeensplitintoitsconstituentfields,andoneor
moreofthosefieldshasbeenmodified(oranewrecordhasbeen
created),itneedstobereassembledintoarecordforputtingbackin
the${db[]}array,whichwillbewrittenbacktoafile.
HowItWorks
Eachfieldisprintedusingprintf,followedbythedelimiter,and
storedinavariable.Thefinal,trailingdelimiterisremoved.Ifthe
delimiterisaspaceoratab,itmaybenecessarytousea
placeholderforemptyfields,sincemanyutilities(including
split_record)willinterpretmultipleconsecutivewhitespace
charactersasasingledelimiter.
Usage
_put_record[-nC]"${ARRAY[@]}"##
storerecordin$_PUT_RECORD
put_record[-nC]"${ARRAY[@]}"##
printtherecord
Thefirstformofthefunctionstorestherecordinavariable,and
thesecondprintstherecord.Ifthe-noptionisgiven,Cisinsertedin
emptyfields.Hereisatypicaluseofthefunctions:
_put_record"${record_vals[@]}"
db[$rec_num]=$_PUT_RECORD
TheScript
_put_record()
{
localNULL=
localIFS=${DELIM:-$TAB}
case$1in
-n*)NULL=${1#-n}##nospaceaftertheoption
letter
shift
;;
-n)NULL=$2##spaceaftertheoptionletter
shift2
;;
*)_PUT_RECORD="$*"
return
;;
esac
_PUT_RECORD=$(
forfieldin"$@"
do
printf"%s${DELIM:-$TAB}""${field:-$NULL}"
done)
}
put_record()
{
_put_record"$@"&&printf"%s\n""$_PUT_RECORD"
}
12.6put_csv—AssembleFieldsintoaCSV
Record
AssemblingaCSVarrayinvolvesenclosingnon-numericfieldsin
quotationmarksandinsertingcommasbetweentheconcatenated
fields.
HowItWorks
Eachfieldistestedtoseewhetheritcontainsnon-numeric
characters(theperiodandminussignareconsiderednumeric).If
onedoes,itisenclosedinquotationmarksbeforebeingaddedtothe
record.
Usage
_put_csv"${ARRAY[@]}"##storerecord
in$_PUT_CSV
put_csv"${ARRAY[@]}"##printthe
record
Thearrayisplacedonthecommandlineasargumentstoeither
formoftheput_csvfunction.Thesyntaxusedinthepreceding
examplespresentseachelementofthearrayasasingleargument,
evenifitcontainswhitespace.
TheScript
_put_csv()
{
forfieldin"$@"##loopthroughthefields(on
commandline)
do
case$fieldin
##Iffieldcontainsnon-numerics,enclose
inquotes
*[!0-9.-]*)
_PUT_CSV=${_PUT_CSV:+$_PUT_CSV,}\"$
;;
*)
_PUT_CSV=${_PUT_CSV:+$_PUT_CSV,}$fi
;;
esac
done
_PUT_CSV=${_PUT_CSV%,}##removetrailingcomma
}
put_csv()
{
_put_csv"$@"&&printf"%s\n""$_PUT_CSV"
}
12.7db-demo—ViewandEditaPasswordFile
Tomaintainadatabase,ausermustbeabletomodifyindividual
fieldsinarecord,andaddanddeleterecords.
HowItWorks
Useload_dbfromtheshdb-funcslibrarytoloadtheentirefileinto
anarray.Aloopdisplaysonerecordatatime,allowingtheuserto
selectfromoptionsdisplayedonamenubar,asshowninFigure123.Fordemonstrationpurposes,thisscriptmakesacopyofthe
passwordfilesothatitcanbeeditedsafely.(Idon’trecommendit,
butyoucanusethisscripttoedityour/etc/passwdfile.)
Figure12-3. Dataentryusingdb-demo
Usage
db-demo
Theusercanmovethroughthefileonerecordatatime,orgoto
anyrecordbyselectinganumberafterpressingg.Pressinga
numberfrom1to7bringsupaprompttoenteranewvalueforthe
fieldatthebottomofthescreen.Pressingtheuparrow(orCtrl-P)
willbringupthecurrentvalue,whichmaybeedited.
TheScript
#!/bin/bash
.screen-funcs-sh##loadscreen
manipulationfunctions
.shdb-funcs-sh##loaddatabase
functions
Bychangingthevaluesinthedb_initfunction,thisscriptcan
beusedwithotherdatabases.Justchangingthefilenameand
changingthevaluesinthelabels[]arraywillallowanothercolonseparatedfiletobeused.Formoregeneraluse,command-line
optionscouldbeusedtospecifytheparameters.Iftherearemany
morethansevenfields,theshow_recordfunctionmayhavetobe
adjusted.
db_init()
{
DELIM=:##fielddelimiter
cls##clearthescreen
##We’llworkwithacopyofthepasswordfile
dbfile=$HOME/etc_passwd##datafile
[-f"$dbfile"]||cp/etc/passwd"$dbfile"
load_db"$dbfile"##storefileinarray
num_records=${#db[@]}##numberofrecords
rec_num=0##counter
cols=$COLUMNS##justforthesakeof
brevity
labels=("Username""Password""UserID""GroupID"
"Fullname"
"Homedirectory""Shell")
}
Usedtoprintamessageinreversevideoacrossthewidthofthe
screenorwindow,thisfunctioncouldeasilybeaddedtostandardfuncsorscreen-funcs.
reverse_bar()
{
##Settoboldreverse
set_attr$bold$reverse
##Printreversebarandmessage
printf"%${cols}.${cols}s\r%s""""$*"
##Removeboldandreverse
set_attr0
}
Theshow_recordfunctionprintsarecordontothescreen.When
dealingwithadatabasethathasmorefieldsperrecord,itmaybe
necessarytochangethelinespacing(lsp)from2to1.Forverylong
records(suchasintheTorontoFree-Netdatabaseformshownatthe
beginningofthechapter),adifferentfunctionwillbeneeded.
show_record()
{
lw=15##Labelwidth
lm=3##Leftmargin
lsp=2##Line
spacing
fmt="%${lw}.${lw}s[$B%d$NA]:%s$cle"##Format
stringforfields
printat11##positioncursorattopleft
ofscreen
##Displayrecordnumberandtotalnumberonreverse
bar
reverse_bar"Record$(($rec_num+1))of
$num_records$db_mod"
field=0
while[$field-lt${#labels[@]}]
do
printat$(($field*$lsp+3))$lm
printf"$fmt""${labels[$field]}"\
$(($field+1))\
"${record_vals[$field]}"
field=$(($field+1))
done
menubar=$(($field*$lsp+5-$lsp))
printat$menubar1
##Printuseroptionsonreversebar
reverse_bar\
"[n]ext[b]ack[g]oto[1-7]Edit[a]dd[d]elete
[s]ave[q]uit"
prompt_line=$(($menubar+2))
prompt_col=$(($lm+$lw-8))
printat$prompt_line$prompt_col"$cle"
}
Themainloopofdb-demowaitsforsingle-keyuserinputand
performsactionsbasedonthekeypressed.
db_main()
{
db_init##Initializethedatabase
while:##loopuntiluserquits
do
##breakthecurrentrecordintoitsconstituent
fields
split_record"${db[$rec_num]}"
show_record
##readasinglekeystroke
read-sn1-p"Select:"x
case$xin
q)break;;##quit;TODO:addquerytosave
ifmodified
a)##Addnewrecord
rec_num=${#db[@]}
db[${#db[@]}]="$DELIM$DELIM$DELIM$DELIM$D
num_records=${#db[@]}
db_mod=*##Modified
flag
;;
b)##movetopreviousrecord(wraparound
ifatfirstrecord)
rec_num=$((($rec_num+$num_records-
1)%$num_records))
;;
d)##Deletecurrentrecord
unsetdb[$rec_num]
db=("${db[@]}")
num_records=${#db[@]}
db_mod=*##Modified
flag
;;
g)##displayaspecificrecord(bynumber)
read-ep"Recordnumber:"rec_num
##checkforvalidrecordnumber
if[$rec_num-gt$num_records]
then
rec_num=$(($num_records-1))
elif[$rec_num-le1]
then
rec_num=0
else
rec_num=$(($rec_num-1))##arrays
countfrom0
fi
;;
s)##Savethedatabase
printf"%s\n""${db[@]}">"$dbfile"
db_mod=##Clearmodifiedflag
;;
[1-7])##Userselectedafieldnumber;
promptfordataentry
in_field=$(($x-1))
##Placefield’scurrentvaluein
history
history-s"${record_vals[$in_field]}"
##Placethecursorattheentryline
andclearit
_getline"${labels[$in_field]}"INPUT
##Ifsomethinghasbeenentered,and
itisnotthesameasbefore
##thenreplacetherecordwiththe
modifiedversion
[-n"$INPUT"]&&["$INPUT"!=
"${record_vals[$in_field]}"]&&{
record_vals[$in_field]=$INPUT##
Storeentryinfield
db_mod=*##
Modifiedflag
_put_record
"${record_vals[@]}"##Rebuildrecord
db[$rec_num]=$_PUT_RECORD##
Placenewrecordintoarray
}
printat$prompt_line1"$cle"
;;
*)##displaynextrecord(wraparoundifat
end)
rec_num=$((($rec_num+1)%
$num_records));;
esac
##checkforvalidrecordnumber
if[$rec_num-ge$num_records]
then
rec_num=$(($num_records-1))
elif[$rec_num-lt1]
then
rec_num=0
#else
#rec_num=$(($rec_num-
1))##arrayscountfrom0
fi
done
printf"\n\n"
}
Afterallthefunctionshavebeendefined,allthescriptneedsto
doiscallthemainloop:
db_main##Enterthemainloop
PhoneBase:ASimplePhoneNumberDatabase
Whiletherearemanydatabasesforcontactinformation,phone
numbers,andotherbasicinformationaboutpeople,mostarefar
morecomplicatedthannecessary.ARolodexmaybeefficientona
physicaldesktop,butitdoesnottakefulladvantageofthe
capabilitiesofamoderncomputer.Formostuses,alloneneedsisa
simpletextfilewithunstructuredrecords,andmethodstosearch
for,add,andremoverecords.
HowItWorks
Simplicityandeaseofusearethehallmarksofthislittledatabase
system,PhoneBase,whichwasbornonmyAmigacomputerinthe
1980s,grewupunderLinuxafewyearsago,andisinuseon
variousoperatingsystemstoday.Itsfeaturesincludethefollowing:
Acommand-lineinterfaceeliminateswaitingforaGUIto
load.
Searchingiscase-insensitive;youdon’tneedtoremember
whetheryourcontact’snameisMacdonaldorMacDonald.
Youcanstorearbitraryinformationalongwithnamesand
phonenumbers.
Youcanaddcommentsthatwillnotbedisplayedwhen
searchingthedatabase.
Searchingcanbedoneforanyinformationentered.
Phonebookscanbeeditedwithanytexteditor.
Youcanusemultipledatabasefiles.
UntilIwashalfwaythroughwritingthissection,PhoneBase
hadonlythreecommands,ph,phadd,andphdel.Nowithasfour;I
addedphx,agraphicalinterface.
12.8ph—LookUpaPhoneNumber
TheworkhorseofthePhoneBasesystemisph,thecommandthat
searchesthedatabasefortherequestedname.
Usage
ph[OPTIONS]STRING
Whilephtakesnooptionsofitsown,itwillpassanyoptions,
alongwitheverythingelseonthecommandline,toegrep.Asa
result,thepermissibleoptionsarethosethatyoursystem’segrep
willaccept.Inaddition,youcanusearegularexpressioninsteadof
aSTRING.
Mostofthetime,allyouwouldneedtouseisasimplesearch:
$phcolin
ColinJohnson,C&JSolutions555-8341
ColinMcGregor416-555-1487(TorontoFreeNet)
Ifyouwanttofindnumbersformorethanoneperson,youcan
passmultiplesearchstringswith-eoptions:
$ph-emichael-erosalyn
RosalynJohnson555-0878
MichaelSmith555-2764
MichaelJohnson555-6528,cell555-4070
IfyouwanttofindJohn,butnotJohnson,youcanusethe-w
optionsincemysystemhasGNUegrep:
$ph-wJohn
Bayliss,John 555-6460
Withotherversionsofegrep,youwouldusethis:
ph'\<John\>'
Formoreinformationonotherpossibilities,readtheegrep
manualpage.
Bydefault,phsearches/var/data/phonesand$HOME/.phones,
butthiscanbeoverriddenbyputtingotherfilesinthephbase
variable.Usethistosearch$HOME/contacts:
$phbase=$HOME/contacts
$exportphbase
$phsomeone
TheScript
Thescriptloopsthroughthefilesin$phbase,orthedefault
/var/data/phonesand$HOME/.phones,andbuildsalistofthosefiles
thatdoexist.Ifanyfilesareintheresultinglist,theyaresearched
byegrep.
[-z"$*"]&&exit5##Ifnoquery,exit
##Ifthe$phbasevariableisnotset,
##placethedefaultfilesin$_phbase
forfin${phbase:-/var/data/phones
$HOME/.phones}
do
[-f"$f"]&&_phbase="$_phbase$f"
done
##Searchfileslistedin$_phbase,
ignoringcomments
[-n"$_phbase"]&&egrep-ih"$@"
$_phbase2>/dev/null|grep-v'^#'
12.9phadd—AddanEntrytoPhoneBase
Additionstothedatabasecanbemadewithanytexteditororword
processor(besuretosaveitasatextfile),orwiththephadd
command.
Usage
phadd[OPTION]"DATA"
Withthe-soption,phaddwillsaveDATAtothesystem’sphone
list,/var/data/phones,ratherthantotheuser’sownfile.
phadd-s"J.JonahJameson555-444-3210"
The-foptiontakesthepathtoafileasanargumentand
appendsDATAtothatfile:
phadd-f$HOME/my_phone_list"PerryWhite
324-597-5263"
TheScript
Sinceonlyoneoptioncanbeused,getoptswouldbeoverkill;
instead,acasestatementexaminesthefirstargumenttoseewhether
itisanoption.
case$1in
##Addtothesystemlistinsteadofyourownlist
-s)phlist=/var/data/phones
shift
;;
##Usefilesuppliedonthecommandline
-f)phlist=$2
shift2
;;
-f*)phlist=${1#-f}
shift
;;
##Usepersonallist
*)phlist=$HOME/.phones
esac
##Appendallnon-optionargumentstothe
file
printf"%s\n""$*">>"$phlist"
12.10phdel—DeleteanEntryfromPhoneBase
Beforeusingphdeltodeletearecordfromyourdatabase,itisa
goodideatousephwiththesamepatterntomakesurethatyouwill
deleteonlytherecordorrecordsyoudon’twant.Oryoucanusea
texteditor.
Usage
phdel[-s]PATTERN
Ifyouusethe-soption,phdelwilldeletefromthesystem
phonelistratherthanyourpersonalfile.
TheScript
Aswithphadd,thereisnoneedtousegetoptsforasingleoption.A
temporaryfilestorestheoutputofgrep-vandisthenmovedover
theoriginalfile.
if["$1"="-s"]##Checkforoptionin
firstargument
then
phlist=/var/data/phones##Usethesystem-widelist
shift##Removeoptionfrom
commandline
else
phlist=$HOME/.phones##Usepersonallist
fi
tmpfile=$HOME/phdel$$##Store
modifiedfileintemporarylocation
grep-iv"$1"$phlist>$tmpfile##Remove
entry
mv$tmpfile$phlist##Replace
datafile
12.11phx—ShowphSearchResultsinanX
Window
WhenChrisstartedwritingthisPhoneBasesection,therewereonly
threecommands;halfanhourlater,hedecidedtolookintothe
possibilityofdisplayingtheresultsinanewwindow.Nowthereare
fourcommands,andPhoneBasecanbeusedwithoutaterminal.
Usage
phx[OPTIONS]PATTERN
SincephxjusthandstheoutputofphtoXdialog,alltheoptions
areexactlythesameasforph.Theresultsforthisscriptareshown
inFigure12-4.
Figure12-4. Theouputofphx
$phxsegovia
TheScript
ThisscriptrequirestheXdialogutility.ItisavailableformostUnix
systems,thoughitmaybecalledxdialog.Ifneitherisavailablefor
yoursystem,youmaybeabletofindequivalentutilitiesonthe
Internet.
Xdialog--left--msgbox"`ph"$@"`"00
Summary
Besidespresentingonecomplete,thoughsimple,application
(PhoneBase),thefunctionswehaveprovidedherearethebuilding
blocksforseriousdatabasemanipulation.Thedemonstrationscript
caneasilybecomethebasisforageneral-purposedatabaseviewer
andentryprogramthatacceptscommandlineargumentsforthefile
andthedelimiter.
Ifyouhaveexperiencewithscripting,youwouldknowthepros
andconsofusingascriptbaseddatabasevsusingaRelational
database(RDBMS).However,thewaythingsarecomingaround
fullcircle,thecommonlyusedpopulardatabasesareoftheNoSQL
genre.Thesearesimilartoourscriptdatabasesbutususallyreside
ontheweb/cloudandareaccessedviaRESTinterfaces.Maybein
thenexteditionofthisbook,wemightbeusingaNoSQLdatabase
instead.
Footnotes
1 Atomicity,Consistency,Isolation,andDurability.See
http://en.wikipedia.org/wiki/ACID.
(1) Ontario,Canada
“Whatelseisthere?”afriendaskedwhenIsaidtheWorldWide
WebwasthepartoftheInternetthatIusedleast.IexplainedthatI
usee-mail,FTP,Usenetnewsgroups,andtelnetorSSHfarmore
thanIusetheWeb.DespitethefactthatIuseittheleast,the
WWW,anditsstandardfileformat,HTML,areindispensable.
ThegazillionsofbadlyconstructedpagesontheWebdonot
negatethefactthatHTMListhebestformatforviewingtext(and
illustrations)acrossdifferentplatformsanddifferenthardware
configurations.Awell-constructedpagewillbeaslegibleona
640×480monochromedisplayasona1600×120024-bitcolor
screen,andeverythinginbetween.
AbrowseristheonlytoolyouneedtoviewHTMLfiles,but
whatifyouwanttodissectthemorcreatethemonthefly?This
chaptercontainsscriptsthatbuildaswellaspullapartHTMLfiles.
TherearealotofoptionstodaytoworkwithHTMLandWeb
technologies,suchasPHP,andRuby.Forsomeusers,Bashwould
beconsideredabitoutdatedincomparisontootherlanguagesand
tools.
PlayingwithHypertext:Thehtml-funcsLibrary
Mostofthetime,Iusethefunctionsinthehtml-funcslibraryatthe
commandline.WhenviewingthesourceofHTMLpagescreated
byHTMLEditorsorotherprogramsthatwritelonglines,with
manyelementsonaline,thefunctionsplit_tagsmakesthem
legible.ThesefunctionsalsoworkwithXMLfiles.
13.1get_element—ExtracttheFirstOccurrence
ofanElement
Themk-htmlindexscriptthatappearslaterinthischaptercreatesan
HTMLpagewithlinkstoallthefilesinadirectory.Wherever
possible,itwouldhelptoincludemoreinformationaboutafilethan
justitsname.Awell-formedHTMLfilewillcontaina<title>tag.
Youwouldneedamethodofextractingthatelement.
HowItWorks
AnumberoftoolsformanipulatingHTMLfilesareavailableasC
sourcecodefromtheWorldWideWebConsortium(W3C)at
http://www.w3.org/Tools/HTML-XML-utils/.Oneoftheseutilities
isextract,whichprintsallinstancesofagiventagthatare
containedinafile.IfyouplantodoalotofworkwithHTMLor
XMLfiles,Irecommendgettingandcompilingthesetools.
Fortheoccasional,relativelysimpletask,ashellscriptcando
thejob.Theget_elementfunctionwillprintthefirstoccurrenceof
anelement,includingtheopeningandclosingtags,allonasingle
line.
Usage
get_element<FILE
COMMAND|get_element
Afilemustbesuppliedtoget_elementonitsstandardinput,not
asacommand-lineargument,andthetagvariablemustbesettothe
nameoftheelementyouwant.Anemptytagvariablewillreturn
thefirstelement.Forget_elementtowork,anelementmusthave
bothopeningandclosingtags.
Mycurrentdirectoryhasanindexfilecreatedbythemkhtmlindexscript.Wecanextractthetitlewith
$.html-funcs##thisisassumed
foralltheexamplesinthissection
$tag=title
$get_element<index.html
<title>chris/tmp</title>
Togetatagfromaremotepage,onecoulduselynx-source,
wget-O-,orcurl-s,andpipethepagetoget_element:
$curl-s
http://www.apress.com
|tag=titleget_element
<title>APRESS.COM|Books
forProfessionals,byProfessionals...</title>
Theshellpageonmywebsitehasanh2element.Icanretrieve
thatwith
$tag=h2
$lynx-source-nolist
http://cfaj.freeshell.org/shell
|get_element
<h2>TheUnixShell</h2>
Ifnoinstanceoftheelementisfound,get_elementreturns1;
otherwise,itreturns0.
TheScript
get_element()
{
##Readuntiltheopeningtagisfound,andprint
theline
whileread-rline
do
case$linein
*\<"$tag"*)
##Linecontainsopeningtag,
##soremoveanyprecedingtext
line=${line#${line%<"$tag"*}}
case$linein
*\</"${tag}"*)
##Linecontainsclosing
tag,
##soremovetrailingtext
line=${line%${line#*
</"$tag"*>}}
##printlineandreturn
fromthefunction
printf"%s"${line}
"${NL:=`printf"\n"`}"
return
;;
*)##Noclosingtaginthis
line,
##soprintitandmoveon
printf"%s"$line
break
;;
esac
esac
done
##Continuereadingandprintinglines
##uptoandincludingtheclosingtag
whileread-rline
do
case$linein
*\</"${tag}"*)
##Linecontainsclosingtagso
removetrailingtext
line=${line%${line#*</"$tag"*>}}
##printlineandmoveon
printf"%s"${line}"$NL"
break
;;
*)printf"%s"${line};;##Linecontains
text;printit
esac
done
}
Notes
Thisscriptwillhaveaproblemwhenatagisnestedwithinanother
instanceofthesametag.Thisshouldnotnormallybeaproblem,as
mosttagsarenotnestedwithinthemselves.
Whenyouareworkingwithlargefiles,andthetagyouare
lookingforisneartheendofthefile,thefollowingversionof
get_elementmaybefaster.Itusesthesplit_tagsfunction,which
appearsnext.Thisversioncantakeoneormorefilenamesonthe
commandline,orreadthestandardinput.
get_element()
{
split_tags"$@"|
sed-ne"/<$tag/,/<\/$tag/p"-e"/<\/$tag/q"|
tr-d"$NL"
printf"\n"
}
Thepureshellversionofget_elementmaybethefastestwayto
retrieveatitle,ifoneexists,becauseitwillbeatthetopofthefile,
andthefunctionstopsreadingthefileassoonasithasretrievedthe
file.Ontheotherhand,iftheelementismissing,thefunctionwill
readthroughtheentirefile.
13.2split_tags—PutEachTagonItsOwnLine
Formanyfunctions,puttingeachHTMLtagonitsownlinemakes
extractinginformationalotsimpler.Ifeverytagisonalineofits
own,extractingtext,forexample,isjustamatterofgrep-v"<".
HowItWorks
Usingawk’sglobalsubstitutioncommand,gsub,anewlineis
insertedbeforeevery<andafterevery>.
Usage
split_tags[FILE...]
Oneormorefilenamesmaybespecifiedonthecommandline,
orsplit_tagscanreaditsstandardinput.
$html='<ulclass="index">
<li><ahref="my_file">Myfile</a>
</li>
</ul>'
$echo"$html"|split_tags
<ulclass="index">
<li>
<ahref="my_file">Myfile
</a>
</li>
</ul>
Sincetheoutputwillusuallybeprocessedfurther,eliminationof
emptylinesisnotbuiltintothisfunction.Forexample,toprintall
thetagsinafile,theoutputofsplit_tagscanbepipedthrough
grep:
$echo"$html"|split_tags|grepe'<'
<ulclass="index">
<li>
<ahref="my_file">
</a>
</li>
</ul>
Toextractalltheopeningtags,grepfindslineswithanopen
bracketthatisnotfollowedbyaslash:
$echo"$html"|split_tags|grep-e
'<[^/]'
<ulclass="index">
<li>
<ahref="my_file">
Alloftheclosingtagsareprintedwiththispipeline:
$echo"$html"|split_tags|grep-e
'</'
</a>
</li>
</ul>
ThetextofanHTMLfilecanbeextractedbyexcludingalltags:
$echo"$html"|split_tags|grepv-e'^<'-e'^*$'
Myfile
ManymuchmorecomplexutilitiesforconvertingHTMLtotext
areavailableontheWeb(searchforhtml2text)orincludedwith
youroperatingsystem.Forsometasks,theseutilitiesareoverkill;
forothers,theyareinadequate.Ifyoucannotfindautilitythatdoes
justwhatyouwant,thesplit_tagsfunctionmaybethetoolyou
needtorollyourown.
TheScript
Thefirstawkcommandinsertsanewlinebeforeanopenbracket,<;
thesecondinsertsanewlineafteraclosingbracket,>.Thethird
commandremovesconsecutivenewlines,andthefourthprintsthe
resultwithoutatrailingnewline,sothattheonlynewlinesarethose
surroundingatag.Anewlineisappendedattheveryend,andthe
wholethingispipedthroughtrtodeletemultiplespaces.
split_tags()
{
awk'/</{gsub(/</,"\n<")}##Addanewlinebefore
"<"
/>/{gsub(/>/,">\n")}##Addanewlineafter
">"
{gsub(/\n\n/,"\n")##removedouble
newlines
printf"%s",$0}##Printwithout
trailingnewline
##Printfinalnewlineandremovemultiple
spaces
END{printf"\n"}'"$@"|tr-s''
}
13.3html-title—GettheTitlefromanHTML
File
Awell-formedHTMLfilealwayshasatitleelementinitsheader.
Whenbuildinganindexpage,you’llwanttoincludethetitlesfrom
thelinkedpages.
HowItWorks
Theget_elementfunctionwillreturnthetitlewithitstags.This
functionretrievesthetitleelementandremovesthetags.
Usage
html_title[FILE]
Ifafilenameisnotgivenonthecommandline,html_titlewill
readthefilefromitsstandardinput.
OnChris’swebsite,hehasasectiondevotedtothenovelsof
DickFrancis.Tocreatetheindex.htmlfile,heusedthisshortscript
atthecommandline:
$forpagein*_novel.html
do
title=$(html_title$page)
printf"<li><ahref=\"%s\">%s</a></li>\n""$page"
"${title%}"
done
<li><ahref="10lb.Penalty_novel.html">10-lb.Penalty</a></li>
<li><a
href="Banker_novel.html">Banker</a></li>
Theresultingfilelookedlikethis(thoughitwasmuchlonger,as
thepagehasfilesforall37ofFrancis’snovels),beforeaddingthe
restoftheinformationusingatexteditor:
<li><a
href="BloodSport_novel.html">BloodSport</a></li>
<li><a
href="Bolt_novel.html">Bolt</a></li>
<li><a
href="Bonecrack_novel.html">Bonecrack</a></li>
<li><a
href="BreakIn_novel.html">BreakIn</a></li>
<li><a
href="ComeToGrief_novel.html">CometoGrief</a></li>
<li><a
href="Comeback_novel.html">Comeback</a></li>
<li><a
href="DeadCert_novel.html">DeadCert</a></li>
<li><a
href="Decider_novel.html">Decider</a></li>
TheScript
html_title()
{
NWSP="[!\\$SPC\\$TAB]"##Patterntomatchnon-space
characters
##Thetitletagmaybeinupper-orlowercase,or
capitalized,
##thereforewetryallthree.Othermixturesofcase
areveryrare
##andcanbeignored.Ifyouneedthem,justadd
themtothelist
fortagintitleTITLETitle
do
set-f
get_element*<${1:-/dev/tty}|##Extractthe
tagtoasingleline
{
readline
line=${line#*<"${tag}"*>}##Deleteupto
andincludingopeningtag
line=${line%</"${tag}"*}##Deletefrom
closingtagtoendofline
case$linein##Iftheline
stillcontainsanything
*$NWSP*)##otherthan
whitespace
printf"%s"$line##printit,
withoutmultiplespaces,
printf"\n"##andprinta
newline
break
;;
esac
}
done
}
HTMLontheFly:Thecgi-funcsLibrary
CommonGatewayInterface(CGI)programscreatewebcontent
whenthepageisrequested;theycanbewritteninanylanguage.
Apartfromafewplaceswhereextraspeedisnecessary,allmineare
shellscripts.Mostarenodifferentfromanyothershellscript,but
theymayneedafrontendtodealwithinformationpassedonbythe
webserverandtoprovidethenecessarycontextheader.
Thelibraryusesthe_substrfunction,soitmustsourcestringfuncs,whichappearedinChapter3:
.string-funcs
13.4x2d2—ConvertaTwo-DigitHexadecimal
NumbertoDecimal
Thefirststepinconvertinga$QUERY_STRINGvariableorPOSTdatato
ausableformatistoconvertthehexadecimal(hex)sequencesto
characters.Thisinvolvesanintermediatestepofconvertingthemto
decimalintegers.
HowItWorks
Thehexsequencesarealwaystwodigits,soasimplelookuptable
willsuffice.
Usage
_x2d2XX##storeresultin$_X2D2
x2d2XX##printtheresult
Thispairoffunctionsfollowsthesameprotocolasmanyother
suchpairsinthisbook:onestorestheresultinavariable,andthe
othercallsthefirstandprintsthecontentsofthatvariable.
$_x2d2ff
$echo$_X2D2
255
$x2d27F
127
TheScript
x2d2()
{
_x2d2"$@"&&printf"%s\n""$_X2D2"
}
_x2d2()
{
##Splitargumentonthecommandlineinto
individualdigits
x2d2_n1=${1%?}##firstdigit
x2d2_n2=${1#?}##seconddigit
##Lookupvalueoffirstdigit
case$x2d2_n1in
[0-9])_X2D2=$(($x2d2_n1*16));;
a|A)_X2D2=160;;##16*10
b|B)_X2D2=176;;##16*11
c|C)_X2D2=192;;##16*12
d|D)_X2D2=208;;##16*13
e|E)_X2D2=224;;##16*14
f|F)_X2D2=240;;##16*15
*)_X2D2=;return5;;
esac
##Lookupvalueofseconddigitandaddtovalueof
firstdigit
case$x2d2_n2in
[0-9])_X2D2=$(($_X2D2+$x2d2_n2));;
a|A)_X2D2=$(($_X2D2+10));;
b|B)_X2D2=$(($_X2D2+11));;
c|C)_X2D2=$(($_X2D2+12));;
d|D)_X2D2=$(($_X2D2+13));;
e|E)_X2D2=$(($_X2D2+14));;
f|F)_X2D2=$(($_X2D2+15));;
*)_X2D2=;return5;;
esac
}
13.5dehex—ConvertHexStrings(%XX)to
Characters
Webapplications,suchasserversandbrowsers,transmit
nonalphanumericcharactersastwo-digithexadecimalcodes
precededbyapercentsign(%).Thesecodesneedtobetranslatedto
theircharacterequivalents.
HowItWorks
Thestring-funcsandchar-funcslibraries(fromChapter3)
providethemeanstochopupthestringandthenconvertadecimal
numbertoanASCIIcharacter.
Usage
_dehexSTRING##storeresultin
$_DEHEX
dehexSTRING##printtheresult
Aspaceisencodedas%20,andquotationmarksare%22:
$dehex
%22The%20quick%20brown%20fox%22
"Thequickbrownfox"
TheScript
dehex()
{
_dehex"$@"&&printf"%s\n""$_DEHEX"
}
_dehex()
{
dx_line=$1
_DEHEX=
while:
do
case$dx_linein
*${dx_prefix:=%}[0-9A-Fa-f][0-9A-Faf]*)
##Storetheportionofthe
string,upto,
##butnotincluding,thehex
string,in$left
left=${dx_line%%${dx_prefix}[09a-fA-F][0-9a-fA-F]*}
##Everythingexcept$leftand
thepercentsign
##isstoredin$right
right=${dx_line#${left}?}
##Thetwohexdigitsarethe
firsttwocharacters
##of$right;extractthemwith
_substr
_substr"$right"12
##Removethehexdigitsfrom
$right
right=${right#??}
##Convertthehexdigitsto
decimal
_x2d2"$_SUBSTR"
##Add$leftandtheconverted
decimalnumber
##totheresultstring,
$_DEHEX
_DEHEX=$_DEHEX$left$(chr-n
"$_X2D2")
##Store$rightin$dx_linefor
furtherprocessing
dx_line=$right
;;
*)##Nomorehex;addrestofline
to$_DEHEX,andexit
_DEHEX=$_DEHEX$dx_line;break;;
esac
done
}
13.6filedate—FindandFormattheModification
DateofaFile
Oneofthemostannoyingthingsonmanywebsites(animations,
frames,andadsareinadifferentclassaltogether!)istheabsenceof
adateforanarticle.Doesthearticle’sinformationstillapply?To
helpwebsitevisitorsdetermineapage’stimeliness,consider
includingthedatesomewhereonthepage.Howdoyoudothat?
HowItWorks
Thisfunctionuseslsandamethodofconvertingthenameofa
monthtoanumberthatwasnotshowninChapter8,whichdealt
withdates.
Usage
filedate
FILE
Theactualformattingisdictatedbythecontentsofastylesheet;
filedatesuppliesthenecessarytags:
$filedateindex.html
<spanclass="filedate">index.html
updated2004-12-11</span>
TheScript
filedate(){
##Splitthedateandplaceintothepositional
parameters
set--$(ls-l"$1")
##Extractthemonthstringuptothegivenmonth
mm=JanFebMarAprMayJunJulAugSepOctNovDec
idx=${mm%%$6*}
##Calculatethenumberofthemonthfromthelength
of$idx
month=$(((${#idx}+4)/3))
day=$7
case$8in
##Iftheoutputoflsshowsthetimeinstead
oftheyear,
##calculatewhichyearitshouldbe
*:*)eval$(date"+y=%Ym=%m")
[$month-gt$m]&&year=$(($y-1))
||year=$y;;
*)year=$8;;
esac
printf"<spanclass=\"filedate\">${9%.cgi}updated"
printf"%d-%02d-%02d</span>\n"$year$month$day
}
CreatingHTMLFiles
EverywebsiteIhavecreatedsinceIstartedwritingthemin1996
hasmadeuseofshellscriptstogenerateatleastsomeofthepages.
Apagemaybestatic—createdandthenleftforposterity—orit
maybecreateddynamicallywhenareaderrequestsit.Athird
categoryissomewhereinbetween:thepageisupdatedonlywhena
changeincircumstanceswarrantsit.Generationofthethirdtypeis
triggeredbyarequestviatheWeb,acommandfromaremoteor
localshell,oracronjob.Thefirstscriptinthissectionisofthe
thirdtype;itcreatesanindexofthefilesinadirectory.
13.7mk-htmlindex—CreateanHTMLIndex
Adirectorylistingasproducedbyawebserverwhenthereisno
index.htmlfileisnotveryattractiveandisdisabledbymanysites.
WhileitiseasytocreateasimpleHTMLindexforadirectory,
producingastandards-compliantpageisalittleharder.
Thissimplescriptproducesaverybasicindexfile:
forfilein*
do
printf"<ahref=\"%s\">%s</a><br>\n""$file""$file"
done>index.html
Abetterscriptwouldproduceaproperheaderforthefile.It
wouldidentifythetypeoffile,andperhapsevenextractatitlefrom
thefileitself.Itshouldbeabletocreateindexesinanentire
directorytree.
Tip
Analternativewayistousetheforfilein`ls`command,
whichwouldlistallthefiles.
HowItWorks
Iftheformattingisdonebyacascadingstylesheet(CSS),thescript
canassignaclasstotheentryforeachfile.Youhavetheoptionto
includedotfiles,orleavethemout;todescendintosubdirectories,
orjustindexthecurrentdirectory;aswellastospecifyvarious
otherfeatures.
Usage
mk-htmlindex[OPTIONS][DIR]
Bydefault,mk-htmlindexwillindexthecurrentdirectory,
identifyingfiletypesbyextension.Theheaderwillincludethe
user’snameandthecreationdateandtime;thenameofthe
directorywillbecomethetitle.Forcertaintypesoffiles—HTML,
XML,PostScript,andsomeshellscripts—itwilltrytofindatitleor
description,anduseitinthefile’sentry.Theoptionsare
-d:Includedotfilesintheindex.
-f:Ifnotitleisgeneratedforthefile,usethefilecommandto
getthetypeoffileandusethatinsteadofatitle.
-iFILE:UseFILEasthenameoftheindexfile.
-l:Includetheindexfilebeingcreatedintheindexitself.
-R:Recursivelydescendthedirectorytree.
-v[v]:Useverbosemode.Additionalvsincreasethelevelof
verbosity.Atpresent,twolevelsareimplemented:inlevel1,
thenamesofanydirectoriesenteredinrecursivemodewillbe
printed,andinlevel2,thenamesofallfileswillbeprintedas
theyareprocessed.
Theworkingdirectoryforthischaptercontains
$ls
4711ch16d1.docindex.cssinfo.cgipretext-sh
cgi-funcs-shindex.htmlmkhtmlindex-shtext2html-sh
html-funcsshindex.htmlxmkhtmlindex-shtoc.html
ThedefaultHTMLfileproducedbythescriptisasfollows
(longlineshavebeenwrapped):
$mk-htmlindex
<!DOCTYPEHTMLPUBLIC"-//W3C//DTD
HTML4.01Transitional//EN">
<htmllang="en">

<head>
<metahttp-equiv="Content-Type"
content="text/html;charset=ISO-8859-1">
<title>Documents/ch16</title>
<linkrel="stylesheet"type="text/css"href="index.css"
</head>
<bodyclass="html">
<spanclass="h2">Listingof
/home/chris/Documents/ch16</span>
<divclass="main">
<ul>
<liclass="doc"><a
href="4711ch16d1.doc">4711ch16d1.doc</a></li>
<liclass="shell"><ahref="cgi-funcs-sh">cgi-funcssh</a></li>
<liclass="shell"><ahref="html-funcs-sh">htmlfuncs-sh</a></li>
<liclass="symlink"><ahref="index.css">index.css
</a></li>
<liclass="html"><ahref="index.html">index.html—
<span
class="title">Documents/ch16</span></a></li>
<liclass="symlink"><a
href="index.htmlx">index.htmlx</a></li>
<liclass="cgi"><ahref="info.cgi">info.cgi</a>
</li>
<liclass="shell"><ahref="mk-htmlindex-sh">mkhtmlindex-sh—
<spanclass="title">"Createindex.htmlfordirectory"
</span></a></li>
<liclass="shell"><ahref="mkhtmlindexsh">mkhtmlindex-sh—
<spanclass="title">"Createindex.htmlfordirectory"
</span></a></li>
<liclass="shell"><ahref="pretext-sh">pretext-sh
</a></li>
<liclass="shell"><ahref="text2html-sh">text2htmlsh</a></li>
<liclass="html"><ahref="toc.html">toc.html</a>
</li>
</ul>
</div>
</body>
</html>
Thisisthestylesheetthatisinusewiththeseindexes.Itusesa
differentbackgroundcolorforeachtypeoffile(moretypescould
beadded):
.main{
line-height:150%;background-color:#aacccc;
margin-left:5%;margin-right:
5%;margin-top:3em;
color:black;
}
.h2{
width:100%;position:fixed;
top:0;margin:0;
padding-left:5%;border-bottom:1ptsolid
black;
background-color:white;font-size:150%;
line-height:1.5;font-weight:bold;
}
li{width:100%;background-color:
#aacccc;padding-left:1em;}
a{padding-left:4px;padding-right:
4px;}
a:hover{color:red;border:outset;
background-color:white;}
ul{background-color:white;}
.txt{background-color:#dddddd;
color:black;}
.dir{background-color:#bbbbbb;
color:black;}
.cgi{background-color:#999999;
color:black;}
.html{background-color:#888888;
color:black;}
.xml{background-color:#777777;
color:black;}
.pdf{background-color:#666666;
color:black;}
.css{background-color:#555555;
color:black;}
.file{background-color:#eeddff;
color:black;}
.doc{background-color:#aaaaaa;
color:black;}
.shell{background-color:#ffffff;
color:black;}
.title{color:black;font-weight:
bold;}
.ps,.eps,.epsi{background-color:
#ddffcc;}
Thereisaleaningtowardstheuseofshadesofgrayinthisstyle
sheetsincethefiguresinthisbookarenotprintedincolor.
Figure13-1. Webpagecreatedbymk-htmlindex
TheScript
mk_htmlheader()
{
date_vars##fromstandard-funcswhichissourcedby
html-funcs
##Lookupuser’sfullnamein/etc/passwd(see
Chapter15)
name=$(lookup-f:-v5$USER/etc/passwd)
##Usenameofcurrentdirectoryasthetitle
title=${PWD#${PWD%/*/*}}
##FormthenameoftheCSSfilefromthenameof
theindexfile
css_file=${filename%.html}.css
##IftheCSSfileisnotinthecurrentdirectory,
##butitisintheparentdirectory,linktoit
[-f"$css_file"]||
{[-f"../$css_file"]&&ln-s"../$css_file"
.;}
##Printtheheader
printf"%s\n"\
'<!DOCTYPEHTMLPUBLIC"-//W3C//DTDHTML4.01
Transitional//EN">'\
'<htmllang="en">'\
"<!--$Id:$filename,$datestamp,${name:-$USER}$->"\
'<head>'\
"<title>${title#/}</title>"\
"<linkrel=\"stylesheet\"type=\"text/css\"
href=\"$css_file\""\
'</head>'\
"<bodyclass=\"${filename##*.}\">"\
"<spanclass=\"h2\">Listingof$PWD</span>"\
'<divclass="main">'\
'<ul>'
}
mk_htmlfooter()
{
##Closealltheopentags
printf"%s\n""</ul>""</div>""</body>""
</html>"
}
##Defaults;thesemaybechangedby
command-lineoptions
verbose=0##Don’tprint
anyprogressreports
recursive=0##Don’t
descendintodirectories
filename=index.html##Filenamefor
generatedHTMLfile
vopts=##Verbose
options,usedforrecursiveoperation
listfile=0##Excludethe
indexfileitselffromthelisting
dotfiles=##Donot
includedotfiles
usetype=0##Donotuse
filetypeifthereisnotitle
progname=${0##*/}
.html-funcs##Loadfunctions
whilegetoptsvRdi:lfvar
do
case$varin
d)dotfiles=.*;;
f)usetype=1;;
i)filename=$OPTARG;;
l)listfile=1;;
R)recursive=1;;
v)verbose=$(($verbose+1))
vopts=${vopts}v;;
*)die5"Invalidoption:$var";;
esac
done
shift$(($OPTIND-1))
##Changetodirectoryonthecommand
line,ifany
[-d"${1:-.}"]&&cd"${1:-.}"||
exit5
##Set$PWDiftheshelldoesn’t
[-z"$PWD"]&&PWD=$(pwd)
##Toavoidcorruptingapagethat
maybeloadedintoabrowser
##whiletheindexingisinprogress,
itissavedtoatemporary
##file.Thatfileisthenmovedin
placeoftheoriginalfile.
tempfile=$(mktemp$filename.XXXXXX)
{
mk_htmlheader##PrinttheHTMLheaderandtopof
<body>
forfilein*$dotfiles
do
##Ifdotfilesareenabled,thecurrent
directoryandparent
##directorywillshowup.Thescriptignores
them.
case$filein
.|..)continue;;
esac
##Thefilewearecreatingwillbeignored
unless
##the-loptionhasbeenselected
["$file"="$filename"]&&[$listfile-eq0]
&&continue
##Clearvariables
title=
class=
##Inveryverbose(-vv)mode,printthenameof
everyfile
[$verbose-ge2]&&printf"%s\n""$PWD/$file"
>&2
if[-d"$file"]
then
##Recursivemode;descendintodirectories
##bycallingthescriptwiththedirectoryas
theargument
if[$recursive-eq1]
then
##Verbosemode:printnamedirectorybeing
descendedinto
[$verbose-ge1]&&
printf"\nDescendinginto%s..."
"$PWD/$file">&2
##Callthesameprogramwithoptionsfrom
thecommandline
$0-${vopts}Ri"$filename""$file"
fi
##Informationforlisting
class=${class:+$class}dir##$fileisa
directory,soadd"dir"toclass
title=""
file=$file/
else
##Thefile’sextensionisusedtodetermine
theclass;ifthe
##filetypemaycontainatitle,itis
searchedforandused
##inthelisting
case$filein
*.html|*.shtml)
class=${class:+$class}html
title=$(html_title"$file")
;;
*.xml)
class=${class:+$class}xml
title=$(html_title"$file")
;;
*[-.]sh)
title=$(head"$file")
class=${class:+$class}shell
case$titlein
*'#DESCRIPTION:'*)
title=${title#*
DESCRIPTION:}
title=${title%%$NL*}
;;
*description=*)
title=${title#*description=}
;;
*)title=;;
esac
;;
*.ps|*.eps|*.epsi)
title=$(head"$file")
case$titlein
*%%Title:\*)
title=${title##*%%Title:}
class=${file##*.}
;;
*)title=;;
esac
;;
*)
title=
if[-L"$file"]
then
class="$classsymlink"
elif[-x"$file"]
then
class="$classexe"
else
case$filein
*.*)class="$class
${file##*.}";;
*)class="$classfile";;
esac
fi
;;
esac
fi
##Ifatitlehasbeengenerated,use<span
class="">todisplayit
if[-n"$title"]
then
title=${title:+—<span
class=\"title\">$title</span>}
##Otherwisedonothingunlesstheuserhas
requestedthe
##filetypeasadefault.
elif[$usetype-ge1]
then
title=$(file"$file")
title="&mdash${title#*:}"
fi
class=${class#}##Removeleadingspace,ifany
##Printlisting
printf"<liclass=\"%s\"><ahref=\"%s\">%s
%s</a></li>\n"\
"${class:-file}""$file""$file"
\
"$title"
done
##Printfooter
mk_htmlfooter
}>"$tempfile"
chmoda+r"$tempfile"
mv"$tempfile""$filename"
13.8pretext—CreateaWrapperAroundaText
File
Whenatextfileneedstobedisplayedinawebbrowser,butits
formattingneedstoremainthesame,itshouldbewrappedin<pre>
tags.
HowItWorks
Besidesenclosingthefilein<pre>tags,thisscriptinsertsa
standards-conformantheader,translatesopenbrackets(<)to<,and
appendsthecorrectclosingtags.Italsogivesthetextsomestyle,
withmarginsonbothsidesforbetterreadability.
Usage
pretext[OPTIONS][FILE]
Atitlecanbespecifiedwith-t,andanoutputfilecanbe
specifiedwiththe-ioption.IfnoFILEisgiven,thestandardinput
isused.
TheScript
##Bydefault,outputgoestothe
standardoutputwhereitcanbe
##redirectedtoafile
htmlfile=/dev/tty
whilegetoptst:h:opt
do
case$optin
h)htmlfile=$OPTARG;;##outputfile
t)title=$OPTARG;;##title
esac
done
shift$(($OPTIND-1))
{
printf"%s\n"\
'<htmllang="en">'\
'<head>'\
"<title>${title:-$1}</title>"\
'</head>'\
'<body>'\
"<h2>${title:-$1}</h2>"\
'<prestyle="margin-left:10%;margin-right:
10%;">'
sed's/</\</g'"$@"##convert'<'to'<'
printf"</pre>\n</body>\n</html>\n"
}>"${htmlfile}"
13.9text2html—ConvertaTextFiletoHTML
AregulartextfileneedstobeconvertedtoHTML.Itisa
straightforwardfile,withparagraphsseparatedbyblanklines.
HowItWorks
Theperfecttoolforthisjobisawk.Ashortscriptcanreadthrough
thefile,insertingparagraphtagsonblanklines,andconvertingopen
anglebracketsto<.The<blockquote>tagindentsthetextfor
readability.
Usage
text2html[OPTIONS][FILE][>
HTMLFILE]
Atitlecanbespecifiedwith-t,andanoutputfilecanbe
specifiedwiththe-ioption.IfnoFILEisgiven,thestandardinput
isread.
TheScript
##defaultistheterminal
htmlfile=/dev/tty
whilegetoptst:h:opt
do
case$optin
h)htmlfile=$OPTARG;;##outputfile
t)title=$OPTARG;;##title
esac
done
shift$(($OPTIND-1))
{
printf"%s\n"\
'<htmllang="en">'\
'<head>'\
"<linkrel=\"stylesheet\"type=\"text/css\"
href=\"main.css\""\
"<title>${title:-$1}</title>"\
'</head>'\
'<body>'\
"<h1>${title:-$1}</h1>"\
"<blockquote><p>"
##insertclosingandopeningparagraphtagson
blanklines,
##andconvertopenbracketsto<
awk'/^$/{printf"</p>\n<p>\n";next}
/</{gsub(/</,"\\<")}
{print}'"$@"
printf"</p></blockquote>\n</body>\n</html>\n"
}>"${htmlfile}"
13.10demo.cgi—ACGIScript
WhenthePOSTmethodisusedinanHTMLformorparametersare
addedaftertheURLonabrowser’saddressline,theyshowupin
the$QUERY_STRINGvariable.Spacesandsomeothercharactersare
translatedtohexadecimal.Thequerystringneedstobeparsedand
thehextranslatedtoASCIIcharacters.
HowItWorks
ThefirstpartofthescriptusesUnixcommandstoshowthedate
andvariablessetbythewebserver.Thenthescriptparsesthe
informationpassedaspartoftheURL,orinthePOSTactionofa
webform,anddisplaystheresults.
Usage
Iusedatextbrowsertoviewthepage,usingthefollowing
commands:
$x='
http://cfaj.freeshell.org/demo.cgi?book=%22Shell
ScriptingRecipes%22&a=1&666'
$elinks"$x"
WedDec1501:06:00UTC
2004
Youareconnectedfrom:
192.168.0.2
Yourbrowseris:
ELinks/0.9.2rc4(textmode;Linux2.6.8.1-12mdk
i686;74x51)
Youhaveappendedthefollowing
QUERY_STRING:
book=%22Shell%20Scripting%20Recipes%22&a=
VARIABLEVALUE
book"ShellScripting
Recipes"
a1
--666
TheScript
ACGIscriptneedsashebangonthefirstlinetotellthewebserver
whatinterpretertousetoexecutethescript.OnaPOSIXsystem,
#!/usr/bin/envbashwillinvokethebashshellwhereveritis.On
othersystems,itmustbereplacedwiththepathtotheexecutable.It
neednotbebash,butitmustbeaPOSIXshell.
#!/usr/bin/envbash
##Loadfunctions
.cgi-funcs
##Thecontent-typeheadermustbe
followedbyablankline
printf"Content-type:text/html\n\n"
##Printpageheadelement
printf"%s\n""<html>""<head>""
<title>Welcome</title>"\
"</head>""<body>"
printf"%s\n""<center>"
##Printvariousbitsofinformation
date
printf"<h3>Youareconnectedfrom:
%s</h3>\n""$REMOTE_ADDR"
printf"Yourbrowseris:
<br>%s<br>\n""$HTTP_USER_AGENT"
printf"<br>Youhaveappendedthe
followingQUERY_STRING:<br>\n"
printf"<br>%s:\n""$QUERY_STRING"
printf"<br><br>%s\n""<table>
<th>VARIABLE<th>VALUE"
##ParsetheQUERY_STRING
##ElementsofQUERY_STRINGare
separatedbyampersands
IFS='&'
set--$QUERY_STRING
forarg##Loopthroughthe
elements
do
_dehex"$arg
case$_DEHEXin
*=*)##SplitintoVARIABLEandVALUE
var=${_DEHEX%%=*}
val=${_DEHEX#*=}
;;
*)var=;val=$_DEHEX;;
esac
printf"<tr><td>%s</td><td>%s</td></tr>\n""${var:--}""$val"
done
printf"%s\n""</table>"
##Tidyup
printf"</body>\n</html>"
Summary
Asinglechaptercanhardlydojusticetothepossibilitiesofusing
shellscriptsinconjunctionwithHTMLfiles.Nevertheless,thereis
agooddealtochewoninthischapter,andthetoolspresentedlay
thegroundworkformoreadvancedscripts.
(1) Ontario,Canada
Chrislovescomposingcrypticcrosswordpuzzles,teachingchess
andwritingcomputerprograms.Hehasalargecollectionofscripts
tohelpyoutodothosethingswell.
Recordkeepingisatediouschore,andthecomputershouldhelp
medomybookkeepingwithlesseffortthanwithoutit.Usingshell
scriptsatthecommandline,Onecouldenteraninvoiceandmove
ontosomethingelseinlesstimethanitwouldtaketoopena
spreadsheetoranaccountingprogram.
Thetwoscriptsinthischapterformthebasisofabookkeeping
system.Firstprcalc,toconsolidateapocketfulofreceiptsintoa
singleledgerentry.Thescriptallowstheusertocommenteach
entry(ifyoufeelconscientious)andsaveittoafileforfuture
reference.
Allthetransactionrecordsarestoredinafilenamedfortheyear
inwhichthetransactionoccurred—GL.2004andGL.2005,for
example.Mostoftheinvoicesaregeneratedbyotherscripts,and
theyenterthetransactionrecordwithoutanyinterventiononmy
part.Whenacrosswordpuzzleisfinished,onecommand(gt
G200503,forexample)generatesatextfilewiththeclues,
encapsulatedPostScriptfileswiththegridandthesolution,andan
invoice,allofwhichite-mailstotheGoodTimesmagazine.Italso
addsatransactionrecordtomyledger.Whenyouneedtoentera
transactionmanually,youcouldusethesecondscriptinthechapter,
gle.
Note
ThescriptsarebasedonsolutionsthatworkpersonallyforChris.
Yourrequirementsmightbedifferentandtherecouldbeoff-theshelfsolutionsthatmayhelpingeneratinginvoicesand
managingfinances.
14.1prcalc—APrintingCalculator
ThecalcscriptinChapter5calculatesasingleexpressionentered
onthecommandline.Forlongcalculations,suchastotalingapile
ofreceiptsfromtheproverbialshoebox,enteringoneamountper
lineiseasier.Sometimesarecordisneededofalltheentries.A
programthatworkslikeadesktopprintingcalculatorwouldbe
useful.
HowItWorks
Themetaphorofadesktopprintingcalculatorisausefulone,butit
shouldn’tbecarriedtoofar.Acomputeriscapableofdoingfar
morethanaprintingcalculator.Theentriescanbestoredinafile,
andtheycanbecombinedwithothersessions.Thefilescanbe
editedandreprocessedthroughprcalc.Theresults,withorwithout
thehistory,canbestoredinavariableforlateruseinascript.There
areotherpossibilitiesfloatingaroundinmyhead,andnodoubtyou
cancomeupwithmanymorenotthoughtof.
Usage
[COMMAND|]prcalc[>FILE]|[|
COMMAND]
Thebasicoperationofprcalcistoaddtheentriesastheyare
entered.Thesearestoredinahistoryfilewhosenameisprinted
whentheprogramisinvoked.Therunningtotalandapromptare
printedtothestandarderrorchannelatthebeginningofeachline.
Enteringablanklineterminatesthescript,andtheresultisprinted
tothestandardoutput.Inthesesampleruns,theuser’sinputis
printedinbold:
$prcalc
Historyfileis
/home/chris/prcalc.history.27224
0:23
23:14.5
37.5:
37.5
Foranoperationotherthanaddition,thearithmeticoperatoris
enteredbeforethenumber:
$prcalc
Historyfileis
0:666
666:/2
333:*1.333
443.889:%10
44.3889:s
0+666
666/2
333*1.333
443.889*10/100
44.3889:q
44.3889
Thestandardarithmeticoperators,+,-,*,and/,foraddition,
subtraction,multiplication,anddivision,respectively,areaccepted
alongwithx(whenfollowedbyanumber)formultiplicationand%
forpercentage.Thepercentsigncanbeplacedafterthenumber;
either%15or15%willwork.Therearealsoanumberofcommands
thatcanbeenteredattheprompt.Inthelastexample,s(save)andq
(quit)areused.Hereareallthecommands:
h:History;displaytheoperationsenteredinthecurrentsession
onthestandarderrorsothatthehistoryappearsontheterminal
evenwhenthestandardoutputisredirected.
s:Savethehistorybyprintingittothestandardoutput,which
mayberedirectedtoafileoraprinter.IfSTDOUTisnot
redirected,thehistorywillbedisplayedontheterminal.
v:Verbosemode;eachoperationperformedisprintedagainst
therightmargin.
q,x:Exitthescript.
=:PrintthecurrenttotaltoSTDOUT,deletethehistory,andstart
anewhistoryfile.
#:Enteracommenttobesavedinthehistoryfile.Acomment
mayalsobeenteredafteranoperation.
BecausepromptingandongoinginformationissenttoSTDERR,
theresultofaninteractivesessioncanbeplacedinafileora
variable:
$result=$(prcalc)
Historyfileis
0:2.3
2.3:v
2.3:1.5%
0.0345:*
122.3*1.5/
100=0.0345
0.414:3.14159265*
20.0345*12=0.414
6.69719:42##Theuniversalanswer0.414+
3.14159265*2=6.6971
48.6972:h6.69719+42##The
universalanswer=48.6972
0+2.3
2.3*1.5/100
0.0345*12
0.414+3.14159265*2
6.69719+42##Theuniversalanswer
48.6972:
$echo$result
48.6972
Whentheinputisnotcomingfromaterminal,thepromptsare
notprinted:
$printf"%s\n"2.453.79"*2""s"|
prcalc>calc.txt
$catcalc.txt
0+2.45
2.45+3.79
6.24*2
12.48
TheScript
Thesh_readfunctionpromptstheuserforinputandreadsthat
input.Ifthescriptisbeingrununderbash(version2orlater),the
readlinelibraryisused,andfullcommand-lineeditingmaybe
done.Previousentriesmayberecalledinthesamewayastheycan
atthecommandprompt,bypressingtheuparrowkeyorCtrl-P.
sh_read()
{
[-n"$x"]&&printf"%s\n""$x">>$HF
case$BASH_VERSIONin
##Ifthisisbash,takeadvantageofhistory
andcommand-lineediting
[2-9]*|[[0-9][0-9]*)
[-n"$x"]&&{
history-c##Clearthe
history
history-r$HF##Readhistory
fromfile
}
read-ep":"${1:-x}
;;
*)printf":">&2;read${1:-x};;
esac
}
Theinitializationofvariablesandthehistoryfileisplacedina
function,asitmaybecalledwhenthetotalisresetwiththe=
command.
prc_init()
{
##Initializevariables
NL='
'
total=0
PR_N=0
x=
op=
set-f##turnoffpathnameexpansion
##Thehistoryfileisre-createdeverysession,
##orwheneverthetotalisresetwith=
HF=$HOME/$progname.history.$$
n=
##addincrementiffileexists
while[-f"$HF${n:+.$n}"]
do
n=$((${n:-0}+1))
done
HF="$HF${n:+.$n}"
>$HF
[-t0]&&printf"Historyfileis%s\n""$HF">&2
##Ifstandardinputisnotconnectedtoaterminal,
##sendpromptsandprogressinfotothebitbucket
[-t0]||exec2>/dev/null
}
progname=${0##*/}
prc_init
##Themainloopofthescriptreads
theuser’sinput,
##parsestheoperatorandoperand,
##andpassesittoawkinmuchthe
samewayasinthecalcscript.
while:
do
printf"%10s""$total">&2
sh_read
case$xin
*%)##Percentagesigncangoafterthenumber
op=%
num=${x%?}
;;
""|q|x)break;;##Quit,exit,etc...
h|s)##Asubshellisusedtopreventthechange
toIFS,
##ortheredirectionofSTDERR,affecting
therestofthescript.
(
IFS=$NL
[$x=s]&&exec2>&1##Sendhistory
tostandardoutput
printf"%s\n"$history>&2
)
x=
continue
;;
x*)op=*;num=${x#x};;
v)##Toggleverbosemode
[${verbose:-0}-eq1]&&
verbose=0||
verbose=1
continue
;;
[a-zA-Z]*)##Ignoreinvalidinput
x=
continue;;
=)##printresultandresettotalto0
printf"%s\n""$total"
prc_init
continue
;;
*)##Separateoperatorandoperand
op=${x%%[.0-9]*}
num=${x#"$op"}
;;
esac
##Adjustoperatorandoperandforpercentage
calculation
case$opin
*%*)
op=*
num="$num/100"
;;
esac
##Addcurrentoperationtohistory
history="$history$NL$total${op:-+}$num"
PR_N=$(($PR_N+1))
old=$total
total=$(awk"BEGIN{OFMT=\"%f\"
printf\"%s\n\",$total
${op:-+}$num
exit}")
##Inverbosemode,printoperation(againstright
margin)
[${verbose:-0}-ge1]&&{
w=$((${COLUMNS:-80}-1))
printf"%${w}.${w}s\r""$old${op:-+}$num=
$total">&2
}
done
printf"\n">&2
printf"%s\n""$total"
14.2gle—KeepingRecordsWithoutaShoebox
Ihavebeenself-employedformorethan15years,andfor20years
beforethatIusuallydidfreelanceworkevenwhenIhadafull-time
job.Inthedaysbeforepersonalcomputers,Iconscientiouslykept
recordsinaccountbooks.WhenIgotmyfirstcomputer,Ithought
thatitwouldallbesomucheasier.Boy,wasIwrong!
Itrieddifferentaccountingandbookkeepingpackages,butall
seemedlikeevenmoreworkthanwritingtheinformationdownon
paper.Ineverreachedthepointofusefulnesswithanyofthe
packages.Perhapsitwasjustimpatienceonmypart,butevery
pieceofsoftwareItriedrequiredtoomuchtimetosetupbeforeI
couldbeginusingit.
Itriedwritingmyownbookkeepingsystems,butIgotbogged
downinthedetails.ItwasafewyearsbeforeIsolvedtheproblem.
Thiscouldbetrueforothers,despitethemanyoptionsavailable.
Surprisinglythingshavenotchangedmuchintermsofrecord
keepingandashoeboxfullofreceipts.
HowItWorks
Theproverbialshoeboxthatisdeliveredtoone’saccountantevery
yearattaxtimemaybeadequateforsomeself-employedworkers,
butmostneedtokeepbettertrackoftheirfinances.Acomputerized
accountingsystemthatistoodifficultorcumbersometouse
efficientlyisworsethanapenandanaccountbook.Mypreviously
well-kept,handwrittenaccountsdegeneratedintoahalf-forgotten
mess,andmycomputerdidlittleornothingtotakeuptheslack.
Idon’trememberwhatledtothebreakthrough,butin
retrospect,thesolutionIcameupwithseemseminentlylogical,if
notdownrightobvious.AllIneededwasadataformatthatincluded
allthenecessaryinformationaboutatransactionandameansof
enteringthatinformation.Oncetheinformationwasstoredina
structuredformat,otherprogramscouldbewrittentoanalyzeitas
andwhennecessary.
Thetransactionswouldbestoredinatextfile,onetoaline.I
originallyusedtilde-delimitedrecords,butthoseeventuallygave
waytotab-separatedfields.Thefieldshaveundergonesome
transformationsincethefirstimplementation,butthere’snotalotof
difference.InowusethefieldsshowninTable14-1.
Table14-1. FieldsintheGeneralLedgerFile
Field
Description
Transaction Auniquenumbergeneratedsequentiallyfromthelastnumberused.
number
Account
Anabbreviationforthenameoftheaccount;ARforaccountsreceivableand
OSforofficesupplies,forexample.Thefullnamescanbelookedupina
chartofaccounts.
Transaction Thedate,ininternationalYYYY-MM-DDformat,onwhichthetransaction
date
tookplace.
Amount
Theamountofthetransactionexpressedincents,sothatintegerarithmetic
canbeusedforcalculations.
Clientor
supplier
Anabbreviationforthenameoftheclientorsupplier.
Details
Informationaboutthetransaction.Thiscanincludeareferencetoanother
transaction(aswhenacheckisreceivedtopayoffaninvoice)oraninvoice
numberforapayableamount.
Dateof
entry
Thedateonwhichthetransactionwasaddedtothedatabase.
Action
taken
Foraccountsreceivable,thisisusually“e-mailed.”Otheractionsinclude
“cashpayment,”“checkdeposited,”and“invoicereceived.”
Somewhatabbreviated,andwithtabsreducedtotwospaces,
hereisanexcerptfromthefileGL.2004.Thetransactionrecordsfor
invoicesarecreatedautomaticallybythesamescriptsthate-mail
mycrosswordpuzzlesalongwiththeirinvoices:
10411AR2004-01-0150000SS5
Puzzles,Jan20042004-01-01E-mailed
10412RE2004-010150000TLJanuaryofficerent2004-01-01Checksent
10413PR2004-01-2150000SSINV:
104112004-01-22Checkdeposited
10414AR2004-012740000GTPuzzlesG2004032004-01-27E-mailed
10415AR2004-02-0140000SS4
Puzzles,Feb20042004-02-01E-mailed
10416RE2004-02-0150000TLRent
forFebruary2004-02-01Checkmailed
10417OS2004-02111979BBPrinterpaper2004-02-13Debitcard
10418PR2004-02-1840000GTINV:
104142004-02-18Checkdeposited
10419AR2004-022440000GTPuzzlesG2004042004-02-24E-mailed
Youcaneasilyimportthisfileintoaspreadsheet,oryoucan
writescriptstoorganizeandprintthedata.Forextracting
informationfromthisfile,awkistheidealtool.Forexample,tofind
howmuchIamowedinaccountsreceivable,Iwouldusethis:
awk'$2=="AR"{ar+=$4}
$2=="PR"{ar-=$4}
END{
print"AccountsReceivable:"ar/100
}'GL.2004
ToextractallthetransactionsfromApril2004andwritethemto
afilethatcanbeprocessedfurther,Iwouldusethefollowing:
awk'$3
/2004-04/{print}'GL.2004>
2004-02.tmp
Theformathaschangedafewtimesoverthelast15years,andI
startedusingthecurrentversionlastyear.Bythetimeyou’re
readingthis,I’llhavehadtowriteascripttoproducethestatement
ofincomeandexpensesthatthetaxdepartmentwants.
Note
Youmightwanttoensurethatthedatasenttothegovernment
complieswithallrequirements.
Thissystemisnotsophisticatedenoughformostbusinesses,buta
self-employedpersonmaywellneednothingmorethanthisanda
fewreport-writingscripts.
Usage
gle[-t]
Withthe-toption,gleusesatestdirectory($HOME/.testing)
insteadoftheregularsetofbooks($HOME/books).(Itreallyismeant
fortesting,notaclandestinesetofaccounts.)
Wheninvoked,gleclearsthescreenandprintsanemptyrecord,
withthefixedfields,transactionnumberandentrydate,filledin.A
defaulttransactiondate(today’sdate)isalsoentered:
TR#:
10649EntryDate:21Dec2004
Date[1]:21Dec2004
Account[2]:
Amount[3]:0.00
Client[4]:
Details[5]:
Action[6]:
Select1-5,[s]ave,[n]ew[q]uit:
Asinglekeypressbringsupapromptorexecutesacommand.
Thenumbers1to6areforchangingthecorrespondingfieldinthe
currenttransaction,ands,n,andqsavethecurrenttransaction,
bringupanewtransaction,andexittheprogram.Ifthecurrent
transactionhasbeenmodified,norqwillprintapromptasking
whetheryouwanttosavethetransaction,discardit,orcancelthe
command.ThispromptwillalsoappearifyouexitwithCtrl-C.
ThisscriptwillworkinanyPOSIXshell,butifyouareusing
bash2+,thehistorycanbeloadedwithresponsesappropriateforthe
field.Thesearestoredinfilesnamedaccounts,clients,amounts,
details,andactions.Theupanddownarrowkeys(orCtrl-Pand
Ctrl-N)willmovebackandforththroughtheentriesinthefile.The
accountsandclientsfilesaretableswithabbreviationsandtheir
expansions.Thisisacut-downversionoftheaccountsfile:
ARAccountsReceivable
APAccountsPayable
PRPaymentReceivedoninvoice
OSOfficeSupplies
REOfficeRent
IRIncomeReceived
CECapitalExpenditure
PCPettycash
DEDepreciation
Theamountsfileisgeneratedeachtimegleisstarted.The
amountsareextractedfromtheGLfileandsortedbyfrequency,so
thattheamountmostoftenusedappearsfirstwhentheuparrowis
pressed.Thesameisdoneforthedetailsfile.
TheScript
progname=${0##*/}
clear
.screen-funcs
.math-funcs
.date-funcs
Afterthefunctionlibrariesareloaded,gle’sownfunctionsare
defined,startingwithmain.Thelibrariesareloadedfirstincaseany
functionstheycontainareredefined,asisthecasewithcleanup,
whichisinstandard-funcs.Amodifiedversionisusedinthis
script.
main()
{
gle_init##Loadfunctionsandsetprogramwide
variables
while:
do##Endlessloop(exitfromwithin
get_form)
tr_init##Initializetransactionrecord
get_form##Displayrecordandgetuserentry
done
}
Thecurrenttransactionisprintedbytr_form.Thefixedfields
areprintedatthetop,onthesecondline(definedbytheline
variable),andeachchangeablefieldisprintedonitsownline.
Thesearedoublespaced,thespacingbeinggovernedbythevalue
of$linespace.
tr_form()
{
line=2
linespace=2
printat$line1
##Use_show_datefromstandard-funcs(Chapter1)
_show_date"$e_date"
printf"TR#:$B%s$UBEntryDate:$B%s$UB$cle"
"$tr_num""$_SHOW_DATE"
_show_date"${tr_date:=$DATE}"
tr_fieldDate1"$_SHOW_DATE"
[-n"$acct"]&&lookup"$acct""$GLDIR/accounts"
&&acct_name=$_LOOKUP
tr_fieldAccount2"$acct${acct_name:+-
$acct_name}"
_fpmul${amount:-0}.01##Convertcentstodollars
case$_FPMULin
*.?)_FPMUL=${_FPMUL}0;;
*.??|*.*);;
*)_FPMUL=${_FPMUL}.00;;
esac
tr_fieldAmount3"$_FPMUL"
tr_fieldClient4"$client${client_name:+-
$client_name}"
tr_fieldDetails5"$details"
tr_fieldAction6"$action"
info_line=$(($line+$linespace))
entry_line=$(($info_line+$linespace))
printat$info_line1"$cle"
printat$entry_line1"$cles"
}
Thepositioningandprintingofeachlineofthetransactionis
donebytr_field,whichtakesthreearguments:fieldname,
selectionnumber,andfieldcontents.
tr_field()
{
line=$(($line+$linespace))##Movedown
$linespacelines...
printat$line1##...atleft
margin
tr_w=$((${COLUMNS:-80}-20))##spaceonline
forcontentsoffield
printf"%12s[%d]:$B%${tr_w}.${tr_w}s$UB$cle\n$cle""$1""$2""$3"
}
Alltheuserinteractionishandledwithintheget_formfunction,
whichloopsuntiltheusersavesthecurrenttransaction,clearsthe
transaction,orquits.
get_form()
{
while:
do
HF=/dev/null##Resethistory
file
tr_form##Display
transaction
printat$info_line1"$cles"##Clearbottomof
screen
printat$entry_line1"Select1-5,[s]ave,
[n]ew:"
get_key
case$_KEYin
1)##Changetransactiondate
info_data=
prompt="Entertransactiondate:"
sh_read&&_parse_date"$_SH_READ"&&
tr_date=$_PARSE_DATE
;;
2)##Enteraccountinfo_data=$accts
prompt="Enteraccountcode:"
HF=$GLDIR/accounts##Foruse
withbashhistory
sh_read&&acct=${_SH_READ%%$TAB*}
HF=
;;
3)##Enteramount
info_data="Enteramountindollarsand
cents(e.g.,$123.45)"
prompt="Enteramount:"
sort_amounts
HF=$GLDIR/amounts##Foruse
withbashhistory
sh_read&&{
printf"%s\n""$_SH_READ">>
"$HF.all"
_fpmul$_SH_READ100
_round$_FPMULamount=${_ROUND}HF=
}
;;
4)##Enterclient
prompt="Entername(orcode)of
client:"
HF=$GLDIR/clients##Foruse
withbashhistory
sh_read&&{
client=${_SH_READ%%$TAB*}
[-n"$client"]&&
lookup"$client"
"$GLDIR/clients"&&
client_name=$_LOOKUP
}
;;
5)##Enterdetailsoftransaction
prompt="Entertransactiondetails:
"
HF=$GLDIR/details##Foruse
withbashhistory
sh_read&&details=$_SH_READ
;;
6)##Enterdispositionof[paper]record
prompt="Actiontakenwith[paper]
record:"
HF=$GLDIR/actions##Foruse
withbashhistory
sh_read&&action=$_SH_READ
;;
s)save_transaction;;
q|n)##Quit/Newtransaction
k=$_KEY
is_changed&&query_save
tr_init
tr_form
case$kin
q)printf"\r$cles"
break2;;
esac
;;
esac
done
}
Whentheuserselectsneworquit,thescriptcallsis_changed.A
successfulreturnfromthefunctionindicatesthatthetransactionhas
beenmodified.
is_changed()
{
##Recordhasbeenchangedif:
##...theamountisnot0
[${amount:-0}-ne0]&&return
##...thereisanythingintheaccount,client,
details,oractionfields
[-n"$acct$client$details$action"]&&return
##...thetransactiondateisnottoday’sdate,
["$tr_date"!="$DATE"]&&return
return1
}
Ifthetransactionhasbeenmodified,theuserwillbeasked
whetherornottosaveit.Theusermayrespondyestosaveit,noto
discardit,orcanceltoignorethecommandandstayintheloop.
query_save()
{
printat$info_line1"$cles"
printat$entry_line6
printf"%s""Savecurrentrecord[Y/n/c]?"
get_key
printat$info_line1"$cles"
case$_KEYin
n|N);;##fallthroughtoaction
c|C)continue;;##gobacktotopofloop
(i.e.,ignorecommand)
*)save_transaction;;##defaultactionisto
savetherecord
esac
}
Theaccountandclientfieldsareusuallyabbreviationsforthe
fullnameofthefield.Thesearelookedupintheappropriatefiles,
andthefullnameisdisplayed.Thisfunctionisalimitedversionof
thelookupcommandpresentedinChapter12.Italwayslooksupthe
firstfieldandreturnsthesecond.
lookup()
{
code=$1
file=$2
[-f"$file"]&&{
tmp=$(grep"^$code$TAB""$file"|cutf2)
[-n"$tmp"]&&_LOOKUP=${tmp#*$TAB}
}
}
ThenameoftheGLfile,GL.YYYY,isderivedfromthe
transactiondate,usingsplit_datefromthedate-funcslibrary
introducedinChapter8.Blankfieldsarereplacedwithasingle
period.
save_transaction()
{
split_date$tr_dateymd
GL_file=$GLDIR/GL.$y
printf"%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n"\
"${tr_num:-.}""${acct:-.}""${tr_date:-.}"
"${amount:-.}"\
"${client:-.}""${details:-.}"
"${e_date:-.}""${action:-.}"\
>>$GL_file
tr_init
}
Theuser’sinputisreadbysh_read.Thefunctioncanbecalled
withanargumentasthevariabletostoretheinput,butinthisscript
itisnormallycalledwithoutanyargument,andthevalueisstored
in$_SH_READ.Ifthefunctionreturnssuccessfully,thenanentryhas
beenmade,anditcanbeassignedtothepropervariable,withsome
massagingifnecessary.
Iftheshellisbash(version2orlater),thenthecontentsofthe
filenamedin$HFarereadintothehistorybuffer,andmaybe
accessedwiththeupanddownarrowsandedited,justasatthe
commandline.
sh_read()
{
printat$info_line1"$cles$RV"
printf"$B%s$NA""$info_data"
info_data=
printat$entry_line1"$cle"
[-t0]&&sttyecho
case$BASH_VERSIONin
##Ifthisisbash,takeadvantageofhistory
andcommand-lineediting
[2-9]*|[[0-9][0-9]*)
history-c##clear
history
history-r$HF##load
history
printf"$B"##print
bold
read-ep"$prompt"
_SH_READ
printf"$UB"##remove
bold
;;
*)##display$promptandreadinputfrom
user
printf"$UB%s$B""$prompt">&2
read-r_SH_READ
;;
esac
##Ifavariablewasnamedonthecommandline,
assignentrytoit
[-n"$1"]&&eval"$1='$_SH_READ'"
##Failifnothinghasbeenentered
[-n"$_SH_READ"]
}
Whenenteringadollaramount,thehistorybuffercontainsall
thevaluesthathavebeenpreviouslyenteredintotheledger.These
aresortedbyfrequency,sothatthemostoftenusedvaluewillbe
thefirstonethatappearswhentheuparrowispressed.Thissorting
isdoneby
sort_amounts()
{
grep'.'"$HF.all"|##Extractallnonblank
linesfromhistoryfile
sort|##Sortamounts
uniq-c|##Countnumberof
instancesofeachamount
sort-n|##Sortnumericallyby
numberofinstances
awk'{print$2}'>"$HF"##Removecounts,and
storeinhistoryfile
}
Thegle_initfunctionisrunbeforethemainloopisentered.It
initializesthedatevariables,decideswhethertousetherealledger
orthetestversion,andbuildstheamountsandactionsfilesfromthe
existingledger.
gle_init()
{
date_vars
##Ifthe-toptionisgiven,ortheprogramis
calledwiththe-shextension,
##usethetestledgerfile,nottherealone
case$1in
-t)testing=1;;
*)
case$prognamein
*-sh)testing=1;;
*)testing=;;
esac
;;
esac
##DirectorytostoreGeneralLedger
if[-n"$testing"]
then
GLDIR=$HOME/.testing
else
GLDIR=$HOME/books
fi
##Storemost-usedamountsin$GLDIR/amounts.all
##forusewithbashhistory
awk-F"\t"'{printf"%.2f\n",$4/100}
'$GLDIR/GL.$YEAR>
"$GLDIR/amounts.all"
##Storeactionsforusewithbashhistory
cut-f8$GLDIR/GL.$YEAR|sort|uniq-c|sort-n|
awk"{print$2}">actions
}
Newrecordsarebuiltwiththenexttransactionnumberandthe
currentdateastheentrydate.
tr_init()
{
date_vars##Initializedateandtimevariables
[-d"$GLDIR"]||{
##Trytocreatedirectoryifitdoesn’texist
mkdir-p"$GLDIR"
[-d"$GLDIR"]||die5"Couldnotcreate
$GLDIR"
}
acct=##Account
acct_name=##Lookup$acctin$GLDIR/accounts
file
tr_date=##Dateoftransaction
amount=##Amountincents(12345,for$123.45)
client=##Usuallyanabbreviation...
client_name=##...whichcanbelookedupin
$GLDIR/clientsfile
details=##Detailsoftransaction
e_date=$DATE##Dateofentry(today’sdate)
action=##E.g.,e-mailedinvoice,received
check,etc.
HF=##Historyfileforusewithbash
##Transactionnumberisincrementedfromlast
numberinfile
tr_num=$(($(cut-f1 /books/GL*|sort-r|head
-1)+1))
}
AnEXITtrap,setinstandard-funcs,callsthecleanupfunction.
Thisscriptredefinesthatfunctionsothatitcheckswhetherthe
currentrecordhasbeenmodified.Ifithas,theuserisaskedwhether
tosaveordiscardit.
cleanup()
{
is_changed&&query_save
[-t0]&&{
[-n"$_STTY"]&&stty$_STTY
sttysane
}
printf"\r$cles"
exit
}
Afterallthefunctionshavebeendefined,themainfunctionis
called.
main"$@"
Summary
Thebookkeepingneedsdiscussedaresmall,andthegeneralledger
systemdescribedinthischapterissimpleforbothdataentryand
reporting.Thelatterisoftendonewithaone-offscript.Ifyouneed
toexpandthedatabase,addingmorefieldsmeanslessthanadozen
linesofadditionalcode.
BothscriptswillruninanyPOSIXshell,buttheyareenhanced
wheninterpretedbybash2+.
Chapter15:RandomActsof
Scripting
(1) Ontario,Canada
Thegenerationofrandomnumbersistooimportanttobeleft
tochance.
—RobertR.Coveyou,OakRidgeNationalLaboratory
Althoughitmayoftenseemotherwise,computersarevery
predictable;theydoexactlywhattheyaretold(i.e.,programmed)to
do.Randomnessistheantithesisofpredictability,andthismakes
thegenerationofrandomnumberbyacomputerproblematic.
Therearetwogeneralmethodsusedbycomputerstogenerate
randomnumbers:hardwareandsoftware.Ahardwaregenerator
maybeadedicateddevice,noisefromdevicedrivers,orsomeother
source.Thesoftwaremethodusesapseudo-randomnumber
generator(PRNG),socalledbecauseamathematicalformula
producesthenumbers.Therandomnumbersgeneratedinthis
chapterusethesecondmethod,aPRNG,whichisbuiltintoeither
theshellorawk.Thisgeneratorusesthelastnumberproducedto
calculatethenextone,unlessanewseedisgiven.
Ifyoutossacoin92times,youdon’texpectittocomeupheads
everytime,butitwouldbeaperfectlyreasonablerandomsequence,
andcertainlyunpredictable.Ifyourolladiesixtimes,youdon’t
expecteachnumbertocomeupexactlyonce,butthemoretimes
youroll,theclosertheresultshouldbetoanevendistribution
acrossallthepossiblevalues.ThePRNGsinmodernshells,aswell
asthatinawk,produceverygoodrandomsequencesthatcanbe
usedforallbutthemostexactingtasks.
Thischapterpresentsfunctionsforgeneratingoneormore
randomnumbersacrossagivenrange,andscriptsthattake
advantageofthem.
Therand-funcsLibrary
Someshellshavea$RANDOMvariablethatreturnsadifferentrandom
numberbetween0and32,767inclusiveeachtimeitisreferenced.
SincethisvariableisnotrequiredbythePOSIXstandard,thereare
shellswithinthescopeofthisbookthatdonotincludeit.Towrite
scriptsthatareportable,yettakeadvantageofthevariablewhenit
exists,wastherandomfunctionwritten.If$RANDOMproducesrandom
numbers,thenthefunctionisdefinedtouseit.Otherwise,random
numbersaregeneratedbyawk.
Twofunctionsthatmakeuseofrandom,tossandrandstr,are
alsoincludedinthelibrary.
15.1random—ReturnOneorMoreRandom
IntegersinaGivenRange
Whenyouwantarandomnumber,youusuallywantmorethanone,
andusuallywanttheminacertainrange.Forexample,ifyouwant
tosimulaterollingapairofdice,youwouldneedtwonumberseach
between1and6inclusive.Ifyouwantfivecardsfromadeckof
cards,youwillneedfivenumbersfrom1to52.
HowItWorks
Iftheshelldoesgenerateanewrandomnumbereachtimethe
variable$RANDOMisreferenced,thenthatisused.Ifnot,awk’srand()
functioniscalled.Atestwhenthelibraryissourceddetermines
whichversiontodefine.Therequestednumberofvaluesisstoredin
the$_RANDOMvariable.
Theinterfacetothesefunctionsisthesamewhetherornotyour
shellhasa$RANDOMvariable,althoughtwooftheoptionsareignored
whenitdoes.
Usage
random[OPTIONS]
Thefirstfouroptionsareusedbybothversionsofthefunction.
Theydeterminethenumberofvaluestoreturn,theupperandlower
bounds(thedefaultsare0and32,767),andaseedforthePRNG.
-lLOWER:Setthelowestpossiblevaluetobegenerated.The
defaultis0.
$random-l32765;echo
$_RANDOM
32766
-nNUM:Thenumberofvaluestobegeneratedisdefinedby
NUM.Ifthisoptionisnotgiven,onevaluewillbereturnedin
$_RANDOM.
$random-n5;echo
$_RANDOM
101351732652117731
2589
-sSEED:UseSEEDasthebasisforallfollowingcalculations.
Thisisusefulfordebuggingascript,asaseedalways
generatesthesamesequenceofnumbers.
$random-n5-s49152;
echo$_RANDOM
15058183472983613746
985
$random-n5-s666;echo
$_RANDOM
765712002281944940
3905
$random-n5-s49152;
echo$_RANDOM
15058183472983613746
985
-uUPPER:Setthehighestpossiblevaluetobegenerated.When
theshellgenerates$RANDOM,thehighestnumberis(inallthe
shellsIhaveused)32,767;thisisthedefaultforbothversions
ofrandom.
$random-n6-u6;echo
$_RANDOM
653213
Thelasttwooptionsareignoredwhenthefunctionisusingthe
shell’s$RANDOMvariable.
-bNUM:Setthenumberofvaluestostorein$_random.By
generatingasmanynumbersasyouarelikelytoneedonthe
firstcalltorandom,thenumberofexternalcalls(toawk)is
reduced.Thedefaultis200.
$random-rb18-u12;
echo$_random
62111041142120
1113411111
-r:Removeanynumbersstoredin$_random.Thisisusedwhen
adifferentsetofparametersisneeded—forexample,ifthe
storedvaluesareintherangeof1to6,andyouneednumbers
between24and36.
$random-l24-u36-n
4;echo$_RANDOM
621110
Becausethevaluesfromthepreviouscallwerestillstoredin
$_random,nonewnumbersweregenerated.The-roptionwipesout
thosevalues:
$random-r-l24-u36-n4;echo
$_RANDOM
31283336
TheScript
Iftheshellgeneratesrandomnumbersin$RANDOM,thechanceoftwo
successivenumbersbeingthesameisextremelysmall,ifnotnil;it
didn’thappeninseveralmilliontriesinthreedifferentshells.
Therefore,if$RANDOMequals$RANDOM,thefunctionthatusesawkwill
beused.
if["$RANDOM"="$RANDOM"]
then
random()
{
##Setdefaults
rnd_num=1##Numberofvaluesto
generate
rnd_lower=0##Lowestvalueofrange
rnd_upper=32767##Highestvalueofrange
rnd_bank=200##Numberofvaluesto
store
rnd_seed=##Seedforrandomnumber
generator
_N=0
_RANDOM=
##Parsecommand-lineoptions,ifany
case$*in*?*)rand_opts"$@"||return5;;
esac
set--$_random##Useremainderofpreviously
generatednumbers
##Thenumbertobegeneratedmustbeaslargeas
thenumbertobereturned
[$rnd_bank-lt$rnd_num]&&rnd_bank=$rnd_num
##Iftherearenotenoughnumbersstored(in
$_random),generatemore
if[$#-lt"$rnd_num"]
then
rnd_bank=$(($rnd_bank-$#))
set--$_random$(echo""|${awk:-awk}'
{
if(length(seed)>0)srand(seed)
elsesrand()
while(n++<bank)
printf"%s",int(rand()*(upper-
lower+1))+lower
}'bank=$rnd_bankseed="$rnd_seed"
lower=$rnd_lowerupper=$rnd_upper)
fi
##Build$_RANDOMwithdesirednumberof
numbers
while[$_N-lt$rnd_num]
do
_RANDOM="${_RANDOM:+$_RANDOM}$1"
_N=$(($_N+1))
case${1+X}inX)shift;;esac
done
##Assignremainingnumbersto$_randomfor
reuse
_random=$*
}
else
random()
{
##Setdefaults
rnd_num=1##Numberofnumbersto
generate
rnd_lower=0##Lowestnumberofrange
rnd_upper=32767##Highestnumberofrange
rnd_seed=##Seedforrandomnumber
generator
_N=0
_RANDOM=
##Parsecommand-lineoptions,ifany
case$*in*?*)rand_opts"$@"||return5;;
esac
##Seedrandomnumbergeneratorifaseedhas
beengiven
[-n"$rnd_seed"]&&RANDOM=$rnd_seed
rnd_mod=$(($rnd_upper-$rnd_lower+1))
while[$_N-lt$rnd_num]
do
_RANDOM="${_RANDOM:+$_RANDOM}$(($RANDOM%
$rnd_mod+$rnd_lower))"
_N=$(($_N+1))
done
}
fi
rand_opts()
{
OPTIND=1
whilegetoptsb:n:l:u:s:rvar
do
case$varin
b)rnd_bank=$OPTARG;;##Numberofvalues
togenerate
l)rnd_lower=$OPTARG;;##Lowerendof
range
n)rnd_num=$OPTARG;;##Numberofvalues
toreturn
r)_random=;;##Reset
s)rnd_seed=$OPTARG;;##Seedtherandom
numbergenerator
u)rnd_upper=$OPTARG;;##Highestnumberto
generate
*)return5;;##Invalidoption
esac
done
}
Notes
WhenusingrandominashellwithoutitsownPRNG,thefunction
willcontinuetousepreviouslygeneratedvaluesunless$_randomis
emptied(manually,orwiththe-roption),orthe“bank”isneverset
largerthanthenumberofvaluestobereturned.
15.2toss—SimulateTossingaCoin
Tosimulatetossingacoin,youwantafunctionthatsucceedshalf
thetime,andfailstheotherhalf,sothatyoucanbasefurtheraction
onitsreturncode.
HowItWorks
Sincetossingacoinisarandomactionwithoneoftwopossible
results,therandomfunctionwithalowerboundof0andanupper
boundof1willdothetrick.
Usage
toss[NUM]
Theoptionalnumber,NUM,ishowmanytossestoperform.The
returnvalueisdeterminedbythefirstnumber,andtheentirethrow
isstoredin$_RANDOM:
$toss12;echo$_RANDOM
010011011010
Amorecommonuseistotestasingleflip:
$toss&&echoHEADS||echoTAILS
HEADS
TAILS
TAILS
TheScript
Thisfunctionsavesandremovesanypreviouslystoredvalues,sets
therangeofnumberto0and1,and,afterrestoringanyexisting
values,teststheresult.
toss()
{
_toss=$_random##Saveany
previouslygeneratedvalues
random-r-l0-u1${1:+-n$1}##Toss
_random=$_toss##Restoreprevious
values
[${_RANDOM%%*}-eq1]##Test
}
15.3randstr—SelectaStringatRandom
Tocreatearandomword,whetherforapassword,atemporary
filename,oranyotherpurpose,youneedascriptthatwillselectone
letterorwordfromalistofseveral.
HowItWorks
Usingthepositionalparametersasanarray,arandomnumberfrom
1tothenumberofparametersisusedtoselectoneofthem.
Usage
_randstrSTRSTR...##Resultin
$_RANDSTR
randstrSTRSTR...##Resultis
printed
Thisfunctioncanbeusedtochooseamonthatrandom:
$randstrJanFebMarAprMayJunJul
AugSepOctNovDec
Mar
Itcanalsobeusedtobuildarandomword:
$word=_random=
$forlin1234567
do
_randstrabcdefghijklmnopqrstuvw
xyz
word=$word$_RANDSTR
done
$echo$word
izyqjty
TheScript
_randstr()
{
random-l1-u$#
eval_RANDSTR=\${$_RANDOM}
}
randstr()
{
_randstr"$@"&&printf"%s""$_RANDSTR"
}
ARandomSamplingofScripts
Intheolddays,whenaCommodore64gracedmydesktop,Iwasted
manyanhourplayingsomesimple,butnonethelessinteresting
games.Iconvertedthreeofthesetoshellscripts:maxit,an
arithmeticalgamewheretwoplayersselectnumbersonagrid,one
playerselectingfromthecurrentrow,theotherfromthecurrent
column;yahtzee,adicegamethatmembersoftheTorontoFree-Net
canplayonlinebytelnettingtotorfree.net);andtic-tac-toe
(whichIknewfromchildhoodasNoughtsandCrosses).Iwould
haveincludedoneoftheminthisbook,butmyeditorwouldhave
hadafit:theyrangefrom500to1,000lines.Instead,hereareafew
simplerexamplesofusingtherand-funcslibrary.
15.4rand-date—GenerateRandomDatesinISO
Format
Whentestingsomeofmyscripts,Ineededamethodofgeneratinga
lotofrandomdates.Forsomescripts,Ineededtobeabletogivea
rangeofyears.
HowItWorks
Sinceeachdaterequiredrandomnumbersinthreedifferentranges,
ratherthanusingtherandomfunctiontoconstrainthenumbers,the
scriptgeneratesthreenumbersforeachdateandperformsthe
modulusoperationoneachoneitself.
Usage
rand-date[-nNUM][-yYEAR][-Y
YEAR]
The-noptionindicatesthenumberofdatestogenerate,andthe
-yand-Yoptionssetthefirstandlastyearsoftherangetoinclude.
Thedefaultisasingledateintherange1900to2100:
$rand-date
1908-11-02
Thenextexampleproducesthreedatesbetween2000and2005
inclusive:
$rand-date-n3-y2000-Y2005
2003-05-26
2002-04-08
2001-02-06
TheScript
##Loadfunctions
.standard-funcs##Foris_numand
otherfunctions
.rand-funcs
##Defaults
n=1
first_year=1900
last_year=2100
opts=n:y:Y:
do
case$optin
n)is_num$OPTARG||exit5
n=$OPTARG
;;
y)is_num$OPTARG||exit5
first_year=$OPTARG
;;
Y)is_num$OPTARG||exit5
last_year=$OPTARG
;;
*)exit5;;
esac
done
shift$(($OPTIND-1))
##Calculatethemodulusfortheyear
rd_mod=$(($last_year-$first_year+
1))
##Generateeachdate
while[$n-gt0]
do
random-rn3##Get3numbersintherange0to
32767
set--$_RANDOM##Placetheminthepositional
parameters
##Calculateyearandmonth
year=$(($1%${rd_mod#-}+$first_year))
month=$(($2%12+1))
##Findmaximumnumberofdaysinmonth
##(leapyearsarenotacknowledged;seeNotes)
set312831303130313130313031
evalmax=\${$month}
##Calculatedayofmonth
day=$(($3%$max+1))
##PrintdateinISOformat
printf"%d-%02d-%02d\n"$year$month$day
##Countdownto0
n=$(($n-1))
done
Notes
Formypurposes,theexclusionofFebruary29wasnotimportant.If
youwanttousethisscriptandincludeFebruary29inleapyears,
sourcedate-funcs(fromChapter8)atthebeginningofthescript,
andusetheselinestoplacethenumberofdaysinthemonthinmax:
_days_in_month$month$year
max=$_DAYS_IN_MONTH
15.5randsort—PrintLinesinRandomOrder
Theorderofchessplayersonaround-robintournamenttable(a
chartthatshowswhoplayswhominwhichround)ischosenat
random.Ascripttorandomizethenamescouldalsobeusedtodeal
orshuffleadeckofcards,orsettheorderinwhichtoaskaroundof
triviaquestions.
HowItWorks
Thelisttoberandomizedisgiventorandsortonetoaline.By
puttingarandomnumberatthebeginningofeachlineandsorting
onthat,theoriginallines,whetherfromafileorstandardinput,will
beinrandomorderoncethosenumbersareremoved.
Usage
randsort[FILE...]
Therandsortscripttakesnooptions,andreadsthestandard
inputifnofilesaregivenonthecommandline.Here,theworld’s
toptenchessplayersareshuffledrandomlyforatournament
pairingstable:
$names='KasparovAnandTopalov
KramnikLeko
MorozevichAdamsSvidlerBacrotShirov'
$printf"%s\n"$names|randsort
Adams
Topalov
Svidler
Anand
Morozevich
Kramnik
Shirov
Kasparov
Leko
Bacrot
Inthenextexample,adeckofcardsgeneratedbythebrace
expansionfoundinsomeshells(e.g.,bash2+andksh93)isdealtto
fourbridgeplayers:
${
printf"\t%-11s\t%-11s\t%-11s\t
%-11s\n"WestNorthEastSouth
printf"\t%s%-9s\t%s%-9s\t%s
%-9s\t%s%-9s\n"$(
printf"%s\n"\
{1,2,3,4,5,6,7,8,9,J,Q,K,A}\
{Hearts,Spades,Diamonds,Clubs}|
rndsort)
}
WestNorthEast
7Diamonds7Spades2
Clubs9Spades
7Hearts2SpadesJ
HeartsKClubs
6Clubs5DiamondsK
Hearts1Diamonds
ASpades4HeartsQ
Spades3Diamonds
4DiamondsQDiamonds8
Hearts5Hearts
AHearts6Diamonds1
Spades9Hearts
QHearts6HeartsA
Diamonds9Diamonds
8Diamonds5Clubs5
Spades7Clubs
3HeartsKSpades4
SpadesJClubs
AClubsJDiamonds6
SpadesKDiamonds
4ClubsJSpades1
Clubs9Clubs
2Diamonds3Clubs8
Clubs2Hearts
3Spades1Hearts8
SpadesQClubs
TheScript
awk'##Seedtherandomnumber
generator
BEGIN{srand()}
##Putarandomnumberinfrontofeachline
{printf"%.0f\t%s\n",rand()*99999,$0}'"$@"|
sort-n|##Sortthelinesnumerically
cut-f2-##Removetherandomnumbers
15.6randomword—GenerateRandomWords
AccordingtoFormatSpecifications
Thegenerationofarandomseriesofletterscaneasilybedonewith
therandstrfunctionfromtherand-funcslibrarypresentedearlierin
thischapter;anexampleisgiveninthe“Usage”section.Sometimes
Iwantawordthatisalittlemorelikeaword,perhapsusinga
certainsetofletters,orlettersandpunctuation.Thesewordsmaybe
usefulaspasswordsortemporaryfilenames.
HowItWorks
Therandomwordscriptletstheuserselectthenumberofwords,the
charactersorcharacterclassestobeincludedinthewords,anda
singlelengthorrangeoflengths.Thisversionrequiresashellthat
generatesitsownrandomnumbers,butasthereareonlytwoplaces
where$RANDOMisused,itwouldbeasimpletasktoconvertitto
usingtherandomfunctionfromrand-funcs.I’llleavethatasan
exerciseforyou,thereader,ifyoufeelsoinclined.
Usage
randomword[OPTIONS]
Theunadornedcommandrandomwordprintsoneword
containingletters,numbers,andpunctuationmarks,sixtoeight
characterslong:
$randomword
J11}X%
The-noptionspecifiesthenumberofwordstoprint,andthe-l
optionspecifiesthelengthofthewords,eitherasanabsolutevalue
orarange,forexample,toproducethreewordsoffiveletters:
$randomword-n3-l5
NLnnK
nr8Ie
OiiiQ
Fivewordsoffrom5to12characterscanbegeneratedwith
$randomword-n5-l5,12
3U)uLIA
eUvZV
US7Oo1
KIAvBl
a?i6Jcmi
Alistofallowablecharacterscanbespecifiedbyclasswiththe
-aoption.Charactersthatdonotmatchaclassaretakenliterally.
Thecharacterclassesare
c:Lowercaseconsonants
C:Uppercaseconsonants
d:Digits,0to9
l:Lowercaseletters,atoz
p:Punctuationmarks
P:POSIXportablefilenamepunctuation(period,hyphen,and
underscore)
u:Uppercaseletters,AtoZ
v:Lowercasevowels
V:Uppercasevowels
Thefollowingprintstwowords(-n2,usingonlyuppercase
consonants-aC):
$randomword-n2-aC
YHBNJXX
LVTPVL
Thenextcommandprintstwowordsusingnumbers(d),
lowercaseconsonants(c),anduppercasevowels(V):
$randomword-n2-adcV
j3q8OO
2UUA4IOr
The-foptionacceptsaspecificformatstring;thewordswill
followtheformatexactly.Here,theformatisanuppercasevowel,
followedbyaconsonant,vowel,andanotherconsonant,allin
lowercase:
$randomword-n2-fVcvc
Igup
Agif
AnneMcCaffreycouldhaveproducednamesforherdragon
ridersofPernwiththiscommand:
$randomword-n6-f"C’cvc"
H’noc
D’lug
Y’zad
C’rar
G’zig
B’cuz
The-foptiontakesanumberspecifyingthelengthofaformat
stringthatwillbegeneratedrandomly.Allwordswillusethesame
format:
$randomword-n4-F7
CM@t)oI
WO=r?uO
GB)l}dA
XJ(d>cI
TheScript
Verysimilarto_randstrintherand-funcslibrary,_randletter
defaultstoselectingarandomletter(selectedfrombothupper-and
lowercase)ordigit,storingthecharacterin$_RANDLETTER.
_randletter(){
[-z"$1"]&&set--$upper$lower$digits
eval"_RANDLETTER=\${$(($RANDOM%$#+1))}"
}
Thecharactersetpassedto_randletterneedstobeseparated
byspaces.Whenthe-aoptionisusedtoselectacharacterset,
interspaceiscalledtoinsertspacesbetweenlettersiftheyarenot
alreadythere.
interspace()
{
echo$*|sed's/./&/g'
}
_randwordgeneratesarandomword(oflength$rw_len)fromthe
charactersin$charset;thiswordisusedasaformatstringby
_rand_wordwhennoneissupplied.
_randword()
{
rw_len=${1:-8}##Defaultto8
characters
_RANDWORD=##Clearthe
variable
##Buildthestringwithcharacterschosenby
_randletter
while[${#_RANDWORD}-lt$rw_len]
do
_randletter${charset:-$upper$lower$digits}
_RANDWORD=$_RANDWORD$_RANDLETTER
done
}
Theworkhorseofthisscriptis_rand_word,whichselectsthe
charactersetforeachletterinthewordbasedonthecharactersin
theformatstring,$fmt,whichmaybesetwiththe-fcommand-line
option.
_rand_word()
{
##Createformatstringifnotspecified
[-z"$fmt"]&&charset=$format_randword${1:-8}
rp_fmt=${fmt:-$_RANDWORD}
_RAND_WORD=
set-f##Disablefilenameexpansion
while[${#rp_fmt}-gt0]
do
rp_=${rp_fmt#?}##Removefirstcharacterfrom
formatstring
rp_char=${rp_fmt%"$rp_"}##Getfirstcharacter
offormatstring
case$rp_charin##Getrandomletterfrom
appropriatechar.set
c)_randletter$consonants;;##Lowercase
consonants
d)_randletter$digits;;##Digits,0to9
l)_randletter$lower;;##Lowercase
letters,a..z
p)_randletter$punct;;##Punctuation
marks
P)_randletter$ppunct;;##Portable
filenamepunct.
u)_randletter$upper;;##Uppercase
letters,A..Z
v)_randletter$vowels;;##Lowercase
vowels
C)_randletter$CONSONANTS;;##Uppercase
consonants
V)_randletter$VOWELS;;##Uppercase
vowels
##Usethenextcharacterliterally,notasa
classdesignation
\\)rp_fmt=$rp_
rp_=${rp_fmt#?}
rp_char=${rp_fmt%"$rp_"}
_RANDLETTER=$rp_char
;;
*)_RANDLETTER=${rp_fmt%"$rp_"}##Usethe
literalcharacter
;;
esac
_RAND_WORD=$_RAND_WORD$_RANDLETTER##Buildword
rp_fmt=$rp_##Resetformatstringwithoutfirst
character
done
set+f##Turnfilenameexpansionbackon
}
rand_word()
{
_rand_word"$@"&&printf"%s\n""$_RAND_WORD"
}
Severaldifferentsubsetsofthealphabetareprovidedtomake
producingsomewhatrealisticwordspossible.Ifyouwantedtogo
further,youcouldsubdividetheconsonantsintophonetictypes,
suchaslabials,sibilants,andfricatives.
lower='abcdefghijklmno
pqrstuvwxyz'
upper='ABCDEFGHIJKLMNO
PQRSTUVWXYZ'
digits='0123456789'
punct='%#!@?>}|\=-_)(+
*&^$<'
ppunct='.-_'
format='uldulp'##Superseded
below;keptjustincase.
##Thesearetohelpmakeplausible
words
vowels='aeiou'
consonants='bcdfghjklmnpq
rstvwxyz'
VOWELS='AEIOU'
CONSONANTS='BCDFGHJKLMNPQ
RSTVWXYZ'
format='uldulpvcVC'##
Charactersforformatstring
##Defaults
fmt=##Format
string
num=1##Number
ofwordstogenerate
length=##Length
ofword
len_lower=6##Lower
rangeforlengthofword
len_upper=9##Upper
rangeforlengthofword
charset=$lower_$upper_$digits_$punct
##Defaultcharacters
opts=n:f:F:l:a:
whilegetopts"$opts"opt
do
case$optin
##classesofcharactersallowed
a)case$OPTARGin
*[!\\][!\\]*)format=$(interspace
$OPTARG);;
*)format=$OPTARG;;
esac
;;
##-lLL,UUlowerandupperno.ofcharacters
l)len_lower=${OPTARG%[!0-9]*}
case$len_lowerin*[!0-9]*)exit5;;esac
len_upper=${OPTARG#*[!0-9]}
case$len_upperin*[!0-9]*)exit5;;esac
[$len_lower-gt$len_upper]&&{
len_lower=$len_upper
len_upper=${OPTARG%[!0-9]*}
}
;;
##numberofwordstogenerate
n)case$OPTARGin*[!0-9]*)exit5;;esac
num=$OPTARG;;
##formatstring,e.g.,-fucClvVdpp
f)fmt=$OPTARG;;
##Allwordsusesame(random)formatstringof
thegivenlength
F)case$OPTARGin*[!0-9]*)exit5;;esac
charset=$format_randword$OPTARG
fmt=$_RANDWORD
;;
esac
done
##Generaterequestednumberofwords
while[$num-gt0]
do
[$len_upper=$len_lower]&&
length=$len_upper||
length=$(($RANDOM%($len_upper-
$len_lower)+$len_lower))
rand_word$length
num=$(($num-1))
done
15.7dice—RollaSetofDice
Forseveralgames,Ineededtoprintdiceonthescreenatspecific
locations.Thesemaybestandard,six-sideddice,ortheymayhave
more(orfewer)sides.Thenumberofdicethatneedtobeshown
willalsovary,fromonetosixormore.
HowItWorks
Usingscreen-manipulationfunctionsandvariablesfromChapter12,
ablockcanbeplacedanywhereonthescreen,usinganyavailable
colors.Therandomfunction,fromtherand-funcslibraryatthe
beginningofthischapter,suppliesthenecessarynumberofrolls.
Thediceareavailablewithupto15sides,thoughtheyare
inaccuratelydisplayedasifacubehadthatmanyfaces.
Thisscriptmaybeusedonitsownandusedinplaceofphysical
dice,butitismoreademonstrationofhowtoincorporatediceinto
largerscripts.
Usage
dice[OPTIONS]
Thisscriptiscontrolledbyfouroptions,allofwhichtake
arguments.Theyare-bforthebackgroundcolorofthedice,-ffor
theforegroundcolor(thedots),-nforthenumberofdice,and-sfor
thenumberofsidesonthedice.
ForthegameofYahtzee,fivestandarddiceareused,asshown
inFigure18-1.
Figure18-1. Arolloffivedice
$dice-n5-s6-bblack-fwhite
TheScript
Thedicearedefinedusingvariablesandfunctionsfromscreenfuncsandscreen-varsinChapter12,allowinganydietobeplaced
anywhereonthescreen.
define_dice()
{
##requiresscreen-vars:$cu_save$cu_restore$cu_d1
$NA
##optional$ds,thecharactertouseforthespots
##providesvariablesd_w,dice{1..15}.
ds=${ds:-o}##Thecharacterusedforthe
spots
ds2=$ds$ds##Twospots
ds3=$ds$ds$ds##Threespots
ds3x="$ds$ds$ds"##Threespotswithspaces
ds4=$ds$ds$ds$ds##Fourspots
ds5=$ds$ds$ds$ds$ds##Fivespots
d_w=5##Width,incharacters,to
allowforspots
##Eachdiecomprises5rows,thefirstandlast
beingblank;
##the$d_fmtstringcontainsthecodetosavethe
cursorposition,
##printarow,restorethepreviouscursor
position,
##andmovedownoneline
d_fmt="$B$cu_save%-$d_w.${d_w}s$cu_restore$cu_d1"
##Eachdie,from1to15isstoredinitsown
variable
dice1=$(printf"$d_fmt"""
"""$ds""""")
"$ds""""$ds""")
"$ds""$ds""$ds""")
"$ds$ds""""$ds$ds""")
"$ds$ds""$ds""$ds$ds""")
"$ds3x""""$ds3x""")
"$ds3x""$ds""$ds3x""")
dice8=$(printf"$d_fmt""""$ds3x""$ds
$ds""$ds3x""")
"$ds3x""$ds3x""$ds3x""")
dice10=$(printf"$d_fmt""""$ds3x""$ds2$ds2"
"$ds3x""")
dice11=$(printf"$d_fmt""""$ds3$ds""$ds3x
""$ds$ds3""")
dice12=$(printf"$d_fmt""""$ds3$ds""$ds2$ds2"
"$ds$ds3""")
"$ds5""$ds3x""$ds5""")
dice14=$(printf"$d_fmt""""$ds5""$ds3
$ds""$ds5""")
"$ds5""$ds5""$ds5""")
dice_defined=1
exportdice1dice2dice3dice4dice5dice6dice7
dice8dice9
exportdice10dice11dice12dice13dice14dice15
dice_definedd_w
}
Theshow_diefunctionprintsadieatthespecifiedpointonthe
screen,usingthedefinedcolors.
show_die()
{
##First2argumentsarerowandcolumn
##Thirdargumentisnumberonfaceofdie
##Requiresscreenfunctionsprintat,set_fgbg
##Optionalvariables:fgandbg(defaultstoblack
andwhite)
printat$1$2
set_fgbg${fg:-$white}${bg:-$black}
case$dice_definedin"")define_dice;;esac
eval"printf\"%s\"\"\$dice$3\""#>&2
}
##Clearvariables
_random=
dice_defined=
##Loadfunctions
.rand-funcs
.screen-funcs
##Defaults
row=3##screenrowtoprint
dice
sides=6##numberofsideson
dice
bg=$white##backgroundcolor
fg=$black##foregroundcolor
num=1##defaultnumberof
dice
whilegetoptsvVhH-:s:n:f:b:c:r:p:
var
do
case$varin
b)get_colour$OPTARG##backgroundcolor
bg=$colour;;
f)get_colour$OPTARG##foregroundcolor
fg=$colour;;
n)num=$OPTARG;;##numberofdice
s)sides=$OPTARG;;##numberofsidesoneachdie
esac
done
shift$(($OPTIND-1))
##Clearscreentodefaultcolors
printf"$NA"
clear
##Generaterandomnumbersforall
thedice
random-r-l1-u$sides-n$num
##Placedicevaluesinthe
set--$_RANDOM
##Printtherequirednumberofdice
n=0
while[$n-lt$num]
do
column=$(($n*($d_w+4)+2))
show_die$row$column$1
shift
n=$(($n+1))
done
printf"$NA\n\n"
15.8throw—ThrowaPairofDice
Whenyoudon’tneedtheimageofthedice,justthenumber,you
canwriteascriptforthespecificsituation.Oneofthemost
commonrequirementsisforoneormorethrowsofapairofsixsideddice.
HowItWorks
Throwingtwosix-sideddiceisnotthesameasthrowingone
twelve-sideddie.First,therangewithatwelve-sideddieisoneto
twelve;withtwosix-sideddice,itistwototwelve.Second,the
distributionofnumbersisdifferent.Withatwelve-sideddie,each
numberhasanequalchanceofbeingthrown,whereasthereisonly
onepossiblecombinationoftwodiceforeachof2and12(1-1and
6-6),andsixcombinationsfor7(3-4,4-3,5-2,2-5,6-1,and1-6),
makingitthemostlikelynumbertobethrown.Thescripttherefore
usestworandomnumbersforeachthrowandaddstheirvalues.
Usage
throw[NUM]
Bydefault,onethrowismade,butthatcanbechangedby
puttinganumberonthecommandline:
$throw5
5
11
7
7
4
TheScript
.rand-funcs##Loadrandom
functions
sides=6##Numberofsideson
thedice
sep='\n'##Stringtoplace
betweennumbers
num=${1:-1}##Defaultto1throw
ifnonumberoncommandline
##Generatealltherandomnumbers
random-r-l1-u$sides-n$(($num
*2))
##Placenumbersinthepositional
parameters
set--$_RANDOM
while[$#-gt0]
do
printf"%s$sep"$(($1+$2))##Printsumoffirst
twonumbers
shift2##Removetwonumbers
done
Summary
Themathematicalliteraturecontainsawealthofinformationand
discussiononrandomnumbers.Muchofitexplainswhythis
methodorthatmethodisnotgoodenoughtogeneratetrulyrandom
numbers.Somepapersrejectpseudo-randomnumbergenerators
altogether;othersallowthattheyhavetheiruses.
Forpracticalpurposes,however,PRNGsareadequateinthe
majorityofcases,andcertainlyforanythingoneislikelytodoina
shellscript.Thescriptsinthischapterpresentedmethodsof
generatingandmanipulatingrandomnumbersaswellassome
applicationsforthem.
Theywillserveyouwellforday-to-dayuse,butpleasedon’t
usethemforencryptingnationalsecrets!
Chapter16:ASmorgasbordof
Scripts
(1) Ontario,Canada
Thescriptsinthissectionarevaried.Manyareresponsesto
questionsinUsenetnewsgroups,andmaysolveaverynarrow
problem.Somearevalidforsomeoperatingsystemsbutnotothers.
Somescriptsusecommandsthatarenotalwaysinstalled(though
theyaregenerallyavailableforUnixsystems).
Itisofteneasiertowriteascriptforaspecificsituationthanto
writeageneralsolutionthatcanbeusedinmanyplaces.Whilethe
latterismoreefficient,andshouldbeattemptedwheneverfeasible,
oneshouldn’truleouttheone-timescript.PeterSeebachsaysit
well:
Agoodruleofthumbistostartthinkingaboutthedesignofa
utilitythesecondtimeyouhavetosolveaproblem.Don’t
mourntheone-offhackyouwritethefirsttime;thinkofitasa
prototype.Thesecondtime,comparewhatyouneedtodowith
whatyouneededtodothefirsttime.Aroundthethirdtime,you
shouldstartthinkingabouttakingthetimetowriteageneral
utility.1
Thefirstscript,topntail,wentfromaone-offone-linertoa
general-purposeutilityinamatterofhours,withthreepeople
contributingtoit.
16.1topntail—RemoveTopandBottomLines
fromaFile
Aftersavingareportfromanapplication,apostertothe
comp.unix.shellnewsgroupwantedtoremovetheheaderand
footer(thefirstthreelinesandthelastthreelines)fromthefile.He
knewhecouldremovethefirstthreelinesbyusingsed'1,3d',but
didn’tknowhowtodeletethelastthree.
HowItWorks
Thefirstthreelinescanberemovedbyusingsedortail,andthe
GNUutilitytac(that’scatbackward)couldbeused.Afterthefirst
threelinesareremovedwithtail,theoutputispipedtotac,which
reversestheorderofthelines.Thisisthenpipedtotailtoremove
thefirstthreelines(whichwereformerlythelastthree),andfinally
throughtacagaintoputthefilebackinthecorrectorder.Thisis
notaveryelegantsolution,butforaone-timescript,itis
acceptable.
tail+4FILE|tac|tail+4|tac
Anothersolutionwouldbetocountthelinesinthefilebyusing
wc,anduseheadtoremovethelinesfromthebottomandtailto
removethemfromthetop:
head-$((`wc-l<FILE`-3))FILE|
tail+4
IwasponderingaprettiersolutionusingawkwhenBillMarcum,
oneoftheregularsinthenewsgroup,postedhissolution:
awk'NR>6{printlast3[NR%3]}
{last3[NR%3]=$0}'
Alittlelater,JanisPapanagnoupostedageneralizedversionof
Marcum’sscriptthatallowedanynumberoflinestobedeleted:
awk-vf=${1:?}-vr=${2:?}'NR>f+r{print
buf[NR%r]}{buf[NR%r]=$0}'
Withthehardworkalreadydone,thescriptwaspackagedwith
command-lineoptionsforthenumberoflinestoberemoved,anda
conditionalthatusessedwhennolinesaretoberemovedfromthe
tail(withoutit,therewouldbea“divisionbyzero”error).The
scriptalsochecksthatargumentstotheoptionsarevalid,i.e.,that
theyarepositiveintegers.
Usage
topntail[-bN][-eN][FILE...]
Bydefault,topntailremovesonelinefromthetopandoneline
fromthebottomofafile.The-band-eoptionschangethenumber
oflinestoremovefromthetopandbottom,respectively.IfnoFILE
isgiven,topntailreadsfromthestandardinput.
$printf"%s\n"alphabetagammadelta
|topntail
beta
gamma
Toremovelinesonlyfromthebottomofafile,use-b0:
$printf"%s\n"1234567|
topntail-b0-e4
1
2
3
TheScript
b=1##No.oflinestoremovefrom
beginning
e=1##No.oflinestoremovefromend
whilegetoptsb:e:opt
do
case$optin
b)b=$OPTARG;;##Numberoflinestoremove
frombeginning
e)e=$OPTARG;;##Numberoflinestoremove
fromend
esac
done
shift$(($OPTIND-1))
case$b$ein##checkfornon-numeric
charactersin$band$e
*[!0-9]*)exit5;;
esac
if[$e-eq0]
then
sed"1,${b}d""$@"##justremovefromthetop
else
##Thebuf[]arrayisarotatingbufferwhichcontains
thelastNlines
##whereNisthenumberoflinestoberemovedfrom
thebottomofthefile
##Printingstartswhenthelinenumberisequaltothe
sumofthenumber
##oflinestoberemovedfromtopandbottom,and
continuestotheend
##ofthefile;theearliestlineinthebufferis
printed.
##ThelastNlineswillnotbeprinted.
awk'NR>b+e{printbuf[NR%e]}
{buf[NR%e]=$0}'b=$be=$e
"$@"
fi
16.2flocate—LocateFilesbyFilenameAlone
ThetraditionaltoolforfindingafilebynameonaUnixsystemis
find.Unfortunately,itisveryslowwhensearchingalargefile
system.Tomitigatethis,adatabase,regularlyupdatedbyacron
job,isoftenusedtostoreallthefilesonthesystem,andacommand
isprovidedtosearchthedatabase.
Themostcommonsystemincurrentuseislocate(whichmay
bealinktoslocate).Thiscommandsearchesadatabasefora
string,afile-globbingpattern,oraregularexpression.Whena
stringisused,locatewillfindthatstringanywhereinthefullpath
tothefile.Inotherwords,locatebashwillreturn
/usr/share/man/man1/bash.1.bz2and/home/chris/.bashrcaswell
as/bin/bash(andprobablyalotofotherfilesaswell).
Howdoesonelocateafilewithoutgettingallofthefilepaths
thatcontainthename?
HowItWorks
Ifasearchpatterncontainsanunescapedwildcardcharacter(*or?),
thenthepatternmustmatchtheentirepathtoafile.Byputtingan
asteriskatthebeginningofthepattern,thepatternonthecommand
linemustmatchtheendofapathname;inotherwords,itmust
matchthefilename.
Usage
flocateFILENAME
IfFILENAMEcontainsaslash,thepatternprecedingitmustmatch
theentirenameofadirectoryforittobedisplayed.Forexample,
thiswouldfindexecutablesnamedcut:
$flocatebin/cut
/usr/bin/cut
/bin/cut
TheScript
locate"*/$1"
16.3sus—DisplayaPOSIXManPage
Sinceyoumightwanttouseyourscriptsonvarioussystems,
considerusingportableoptionswithUnixcommands.Whilenot
infallible,thePOSIXstandard(whichhasbeenmergedwithSUS)
givesagoodindicationofwhatonecanexpectinmost
implementationsofUnixcommands.ThemanpagesforPOSIX
commandsareavailableontheOpenGroupwebsite.2Ratherthan
rememberingtheURLeachtime,youcouldhaveacommandtodo
thatforyou.
HowItWorks
ThefirsttimeyourequestanyparticularmanpagefromtheOpen
Groupwebsite,thescriptsavesittoalocaldirectoryandusesthat
onallsubsequentrequestsforthatpage.
Usage
sus[-d]COMMAND
SincethepagesaremarkedupwithHTML,thelogicalchoice
forviewingitisawebbrowser.Fromthecommandline,thelogical
browserforthescripttouseislynxorlinks.Iprefertheformer,but
feelfreetochangethescriptifyouprefertheother.
Note
lynxisnotavailableonMacOSXbydefault,youmighthaveto
compileityourselfforuse.
Ifthe-doptionisgiven,lynxisusedtodumptheformattedtextof
thepagetothescreen.
TheScript
.standard-funcs##loadfunctionsfrom
libraryin
Chapter1
dump=0
whilegetoptsdopt
do
case$optin
d)dump=1;;
esac
done
shift$(($OPTIND-1))
sus_ldir=$HOME/work/sus##
DirectoryforstoringHTMLpages
[-d"$sus_ldir"]||##If
directorydoesn’texist
mkdir-p"$sus_ldir"||exit3##
createit
html_file=$sus_ldir/$1.html##
PathtostoredHTMLfile
##LocationofmanpagesonOpenGroup
websitesus_dir=
http://www.opengroup.org/onlinepubs/007904975/utilities
##Ifthefiledoesn’texistlocally,
downloadit
[-f"$html_file"]||wget-O
$html_file$sus_dir/$1.html>/dev/null2>&1
case$dumpin
1)##Dumpfilewithlynx
lynx-dump-nolist$html_file|${PAGER:-less}
;;
*)##Viewfilewithlynx
lynx$html_file
;;
esac
16.4cwbw—CountWordsBeginningWith
Aposterinthecomp.unix.shellnewsgroupaskedhowtocountthe
wordsbeginningwithfinacomma-separatedlist.Theiswhatthe
list,inavariable$A,contained:
A=qad,security,printq,mfgpro,sugroup,f000000m,f000000r,f000000
HowItWorks
Thescriptusesprintfandamodified$IFStoreplacethecommas
withnewlinesandpipetheoutputtogrep-c.
Usage
cwbwSTRING
Withtheprecedingstringinvariable$A,cwbwcountsthenumber
ofwordsbeginningwithf:
$cwbw"$A"
3
TheScript
IFS=,
printf"%s\n"$*|grep-c"^f"
Notes
Analternativescriptusestrratherthanprintftobreakthestring
intoseparatelines:
echo"$@"|tr',''\012'|grep-c"^f"
Thisone-offscriptissimpleenoughtomodifytosuitchanging
requirements.Ifitturnsouttobeofgeneraluse,eitherversionof
thescriptcanbemodifiedtoallowspecificationofdifferent
separatorandsearchcharactersonthecommandline.
16.5cci—Configure,Compile,andInstallfrom
Tarball
Programsdownloadedassourcecodeusuallycomeinpackages
knownastarballs,archivescreatedwithtarandthencompressed
withgzip.Theyusuallyincludeaconfigurescriptthatbuildsa
Makefile.Formostofthesepackages,theprocessisasfollows:
1. Uncompressthetarballwithgunzip.
2. Unarchivetheresultingfilewithtar.
3. Changedirectorytotheonecreatedbytar.
4. Runtheconfigurescript.
5. Compiletheprogramwithmake.
6. Installtheprogram,asroot,withmakeinstall.
Sincethesestepsarethesameforthemajorityofpackages,this
isacandidateforascriptthatdoesitallwithasinglecommand.
HowItWorks
Theonlycomplicationinwritingthisscriptisknowingwhich
directorytocdintoafterunarchivingthesourcecode.Byusing
tar’sverbosemode,thefirstlineoutputwillbethenameofthe
directory.Thescriptusessettostoretar’soutputinthepositional
parameters.Thenameofthedirectorywillbein$1.
Usage
cciPACKAGE.tar.gz
Somepackagenamesendwith.tgzinsteadof.tar.gz.This
scriptwillworkwitheither.
TheScript
gunzip"$1"##
Uncompressthefile
case$1in
*.gz)file=${1%.gz};;##Filewillhaveno
extension
*.tgz)file=${1%.tgz}.tar;;##Filewillhave.tar
extension
esac
##Placethepathtotheextracted
filesinthepositionalparameters
set--$(tar-xvf"$file")
##Thefirstpositionalparameterwill
containthenameofthedirectory,
##socdintoit,andconfigureand
compiletheprogram
cd$1&&./configure&&make&&sudo
makeinstall
Notes
IwrotethisscriptinanswertoaquestionpostedonaLinux
newsgroup,althoughIrarelyuseitmyself.Instead,Iusuallyunpack
thefileatthecommandlineandcdintothedirectorythuscreated.
Usually,thereisaREADMEfilewithinformationaboutthepackage,
anditoftencontainsinstructionsonhowtoprepareandcompileit:
tarxvzfFILE.tgz##requiresGNUtar;
otherwisegunzipthefilefirst
cd<nameofdirectory>##dependsonthe
package
tREADME##'t'isaliasedto
'less-egimQrXF'
Then,dependingonthecontentsoftheREADMEfile,youmay
makeachangeinafile,orreadtheINSTALLfile,orperformsome
otheractionsbeforedoing./configure&&make.
16.6ipaddr—FindaComputer’sNetwork
Address
TheUnixsystemsIusehaveacommandthatshowsnetworking
information.Thecommand,ifconfig,includesthemachine’sIP
addressesinitsoutput.OnmyLinuxsystems,theoutputlookslike
this:
eth0Linkencap:EthernetHWaddr
00:07:95:18:89:2F
inetaddr:192.168.0.49Bcast:192.168.0.255
Mask:255.255.255.0
inet6addr:fe80::207:95ff:fe18:892f/64
Scope:Link
UPBROADCASTRUNNINGMULTICASTMTU:1500
Metric:1
RXpackets:478882errors:3dropped:0
overruns:0frame:4
TXpackets:576878errors:0dropped:0
overruns:0carrier:0
collisions:1513txqueuelen:1000
RXbytes:288668489(275.2Mb)TX
bytes:299036237(285.1Mb)
Interrupt:11Baseaddress:0xd400
loLinkencap:LocalLoopback
inetaddr:127.0.0.1Mask:255.0.0.0
inet6addr:::1/128Scope:Host
UPLOOPBACKRUNNINGMTU:16436Metric:1
RXpackets:614904errors:0dropped:0
overruns:0frame:0
TXpackets:614904errors:0dropped:0
overruns:0carrier:0
collisions:0txqueuelen:0
RXbytes:97049241(92.5Mb)TX
bytes:97049241(92.5Mb)
Theoutputcanbelimitedtoasingleinterface(inthisexample,
eithereth0(IPaddress192.168.0.49)orlo(127.0.0.1).Eachsection
containsthemachine’sIPaddressforthatinterface.Thataddress
maybetheInternetaddress,oraLANaddress,withtheInternet
addresscontrolledbyanetworkaddresstranslation(NAT)gateway.
Acommonquestioninthenewsgroupsis,HowdoIfindthe
machine’sIPaddress?
HowItWorks
Withtheoutputofifconfigstoredinavariable,parameter
expansioncanextracttheIPaddress.Differentsystemsusedifferent
formats,andthisscriptwillgiveresultsforLinux,FreeBSD,and
NetBSD.
Usage
ipaddr[-n][interface]
Ifipaddrisusedwithoutanyoptionorargument,itwillreturn
thefirstIPaddressitfinds.Ifthenameofanetworkinterfaceis
given,ipaddrreturnsthefirstIPaddressforthatinterface.
The-noptiontellsthescripttofindtheInternetaddress,which
willnotbereturnedbyifconfigifthecomputerisonalocal
networkbehindaNAT.Instead,thescriptwilluselynxtogetthe
addressfromaCGIscript(seethenextscript,ipaddr.cgi)onmy
website.
TheScript
if["$1"="-n"]
then##Gettheaddress
externallywithlynx
ip=$(lynx-dump
http://cfaj.freeshell.org/ipaddr.cgi
)
else
if=$1##Nameofinterface(optional)
system=$(uname)##Whatsystemisbeingused?
case$systemin
##SetthestringappropriatetotheOS
FreeBSD|NetBSD)sep="inet";;
Linux)sep="addr:";;
*)printf"System%sisnotyetsupported\n"
"$system"
exit1
;;
esac
temp=$(ifconfig$if)##Gettheinformation
temp=${temp#*"$sep"}##RemoveeverythinguptotheIP
address
ip=${temp%%*}##RemoveeverythingaftertheIP
address
fi
printf"%s\n""$ip"##PrinttheIP
address
16.7ipaddr.cgi—PrinttheRemoteAddressofan
HTTPConnection
MyInternetconnectionhasadynamicIPaddress;itcanchange
anytimemyserviceproviderwants.HowcanIfindthecurrent
networkaddressformyInternetconnection?
HowItWorks
IhaveawebsiteonwhichIcanrunCGIscripts,andtheaddressof
computersthataccessitisstoredinREMOTE_ADDR.Theipaddr.cgi
scriptprintsthecontentsofthatvariable(aftersendingthe
necessaryheader).
Usage
Thescriptmustbeplacedinadirectoryfromwhichtheuserhas
permissiontorunCGIscripts,andtheexecutebitmustbeset.
TheScript
#!/bin/sh
echo"Content-type:text/plain
"
echo"$REMOTE_ADDR"
16.8iprev—ReversetheOrderofDigitsinanIP
Address
FromtimetotimeintheUnixorLinuxnewsgroups,someonehas
obtainedanIPaddresswiththedigitsreversed(forexample,
49.0.168.192insteadof192.168.0.49)andwantstoputthemback
inthecorrectorder.Thesereversedottedquadnumbersareusedby
domainnameserversforreverselook-upstofindthenameofahost
givenitsIPaddress.(Anexplanationofhowitworkscanbefound
athttp://howtos.linux.com/guides/nag2/x-087-2resolv.howdnsworks.shtml.)
HowItWorks
Bychangingthefieldseparator(IFS)toaperiod,setcanassign
eachelementofthedottedquadnumbertoapositionalvariable.All
thatremainsistoprinttheminthereverseorder.
Usage
iprevNN.NN.NN.NN
TheScript
IP=$1##Theaddresstobe
reversedisinthefirstargument
IFS='.'##Setthefield
separatortoaperiod
set--$IP##Splittheaddress
intoitscomponents
echo"$4.$3.$2.$1"##Printtheresult
inthereverseorder
Notes
InsteadofchangingIFS,parameterexpansioncouldbeused:
IP=$1
ab=${IP%.*.*}
cd=${IP#*.*.}
echo"${cd#*.}.${cd%.*}.${ab#*.}.${ab%.*}"
16.9intersperse—InsertaStringBetween
DoubledCharacters
Aposteroncomp.unix.shellaskedhowtoconvertthisstring:
123|456||abc|||cdef||||end
intothisone:
123|456|\N|abc|\N|\N|cdef|\N|\N|\N|
end
I’vehadtodosimilarthingstoadatabasefilewhentheutilities
haveaproblemwithemptyfieldsandinterprettwoormore
consecutiveseparatorcharactersasasingleseparator,ratherthan
boundariesofemptyfields.Forexample,inafilewithtab-separated
fields,thefollowingrecord(withliteraltabcharactersinsteadof
${TAB})wouldcauseproblemsinsomeshellscripts:
field=13023${TAB}${TAB}2002-1103${TAB}20599${TAB}STA
Aftersplittingthisrecordwithset,thereareonlyfourfields
insteadoffive.Aplaceholdermustbeinsertedbetweenconsecutive
tabs.
HowItWorks
Theposteralreadyhadasolution,butwantedamoreefficientone.
Hissolutionwasthis:
catsome_file|sed's/|||/|\\N|\\N|/g'|
sed's/||/|\\N|/g'
Besidestheunnecessaryuseofcat(seeChapter1),two
invocationsofsedareunnecessary,assedcantakemultiple
commandsonthecommandline.Allthatisneededisonecalltosed
withtwocommands(usingthe-eoption).
Thealgorithmcanbegeneralized.Foreverytwoconsecutive
instancesofX,insertYbetweenthem,thendoitagain.Thisallows
anycharacterorstringtobegivenonthecommandlineforeither
element.
Usage
intersperseBOXCONTENTS[FILE...]
TheCONTENTSwillbeplacedinsideallconsecutiveinstancesof
theBOX.IfnoFILEisgiven,interspersewillusethestandardinput.
Forexample,toinsertanunderscoreintheemptyfieldinthe
preceding$fieldvariable,usethis:
$printf"$field"|intersperse"$TAB"
"_"
13023_2002-11-0320599$
STA
TheScript
[$#-lt2]&&exit1##Checkthat
thereareatleasttwoarguments
box=$1##Enclosing
characterorstring
contents=$2##Character
orstringtobeinserted
shift2##Remove
firsttwopositionalparameters
##Twocommandsarenecessary;the
firstwillconvert(forexample)
##"||||"to"|X||X|",andthesecond
willtakecareofanyremainingdoubles
sed-e"s/$box$box/$box$contents$box/g"
\
-e"s/$box$box/$box$contents$box/g""$@"
16.10ll—UseaPagerforaDirectoryListing
OnlyIfNecessary
Whendisplayingalistoffilesinaterminal,youwouldliketousea
pagerwhenthelistislongerthanascreenful,butonlythen.Ifthe
listisshorterthanthenumberoflinesonthescreen,youcanprefer
ittobeprinteddirectly.Whilelessisveryconfigurable,youhave
notbeenabletogetittodothis.The-Foptiontellslesstoexitif
thereislessthanonescreen,butitprintsatthebottomofthescreen
ratherthanstartingatthecurrentcursorposition.
HowItWorks
Thellscriptstorestheoutputofls-linavariableandcountsthe
lines.Ifitismorethanthenumberoflinesonthescreen,itpipes
thevariablethroughless;otherwise,itechoestheresulttothe
screen.
Usage
ll[OPTIONS][FILE...]
Sinceallofitsargumentsarepassedtols,youmayuseany
optionsthatyourversionoflsaccepts.Ifyouwantanyoptionsto
beusedbydefault,assignthemtothels_optsvariableinthescript.
TheScript
IFS='
'
ls_opts=##Assignoptionsas
defaultbehavior;Iuse"-lA"
list=$(ls$ls_opts"$@")
set--$list
if[$#-lt$LINES]
then
printf"%s\n"$list
else
##Yourversionoflessmaynothavealltheseoptions,
soadjusttotaste
printf"%s\n"$list|${PAGER:-less-egimQrXF}
fi
16.11name-split—DivideaPerson’sFullName
intoFirst,Last,andMiddleNames
Aperson’sfullnamemaybejustfirstandlastnames,oritmayalso
containoneormoremiddlenamesorinitials.Whentheentirename
isinasinglevariable,amethodisneededtosplititintoits
components.
HowItWorks
Ifthefirstandlastwordsarethefirstandlastnamesrespectively,
theneverythingelseisthemiddlename(ornames,orinitials).The
nameissplitusingset,whichplacestheelementsofthenamein
thepositionalparameters.
Usage
name-split
Thenameisreadfromthestandardinput.Ifthatisaterminal,
theuserispromptedforaname:
$name-split
Entername:ChristopherFrederick
ArnoldJohnson
Lastname:Johnson
Firstname:Christopher
Middlename:FrederickArnold
Ifeitherthefirstorlastnamecontainsaspace,itmustbe
escaped.Escapingcanbedonebyprecedingthespacewitha
backslash:
$name-split
Entername:RalphVaughan\Williams
Lastname:VaughanWilliams
Firstname:Ralph
Middlename:
Alternatively,theentirenamemaybeenclosedinsingleor
doublequotes:
$name-split
Entername:Alicia"delaRocha"
Lastname:delaRocha
Firstname:Alicia
Middlename:
TheScript
##Usethebashreadlinelibraryif
it’savailable
[2-9]|[0-9][0-9])read_opt='-rep';;
*)read_opt=-r;;
esac
##Thepromptwillonlybedisplayedif
thestandardinput
##isconnectedtoaterminal;bash
doesthisautomatically
##whenread’s-poptionisused;other
shellsneedtotestforit
prompt="Entername:"
[-n"$read_opt"]&&read$read_opt
"$prompt"name||{
[-t0]&&printf"%s""$prompt"
read$read_optname
}
set-f##Turnoff
pathnameexpansion
eval"set--$name"##Putthename
intothepositionalparameters
first=$1##Firstname
is$1
eval"last=\${$#}"##Lastnameis
thelastparameter
shift##Removethe
firstparameter
middle=$*##Middlename
iswhat’sleft,after
middle=${middle%$last}##removingthe
lastname
##Adjustoutputtoyourownneeds
printf"\nLastname:%s\n""$last"
printf"Firstname:%s\n""$first"
printf"Middlename:%s\n""$middle"
16.12rot13—EncodeorDecodeText
Bynomeansacryptographicallysophisticatedcode,rot13is
intendedtomaketextunintelligibleataglance.Itisoftenusedin
theUsenetnewsgroupstoencodetheanswertoapuzzlesothatthe
answerisnotgivenawayimmediately,butcanstillbeeasily
convertedtoplaintext.
HowItWorks
MostUsenetnewsreadershaverot13encodinganddecodingbuilt
in,butforotheruses,trcandothejob.Sincethecodeis
symmetrical(AbecomesM,andMbecomesA),thesamescriptboth
encodesanddecodesthetext.
Usage
tr[FILE...]
Ifnofilesaresuppliedonthecommandline,rot13readsthe
standardinput:
$printf"%s\n""Thequickbrownfox"|
rot13
Gurdhvpxoebjasbk
$printf"%s\n""Gurdhvpxoebjasbk"|
rot13
Thequickbrownfox
TheScript
##Rangesofcharactersareused;[a-m]
expandstoabcdefghijklm
##andwillbereplacedbythe
correspondingcharacterintherange[n-z]
##Eachletterinthefirsthalfofthe
alphabetisreplacedby
##aletterfromthesecondhalf,and
viceversa.
cat"$@"|tr'[a-m][A-M][n-z][N-Z]'
'[n-z][N-Z][a-m][A-M]'
Notes
Ifyouprefertoencodetheargumentsonthecommandlinerather
thanusethestandardinput,usethisscriptinstead:
printf"$*\n"|tr'a-zA-Z''n-za-mN-ZA-M'
16.13showfstab—ShowInformationfrom
/etc/fstab
Inthealt.linuxnewsgroup,someonesuggestedanewformatfor
the/etc/fstabfile,whichtellsthemountcommandhowandwhere
tomountdiskpartitions.Eachdeviceentrywouldbeonsixlines
insteadofonelineasitisnow.Eachlinewouldbelabeled,andthe
entrieswouldbeseparatedbyablankline:
device/dev/hda1
mountpoint/
typeext3
optionsdefaults
dump1
pass1
device/dev/hda2
mountpoint/home
typeext3
optionsdefaults
dump1
pass2
Otherpeoplesuggestedthatthedifficultyofconversiontoand
fromthenewformatwouldbeastumblingblocktoitsadoption.I
hadnostrongfeelingsabouttheformateitherway,butIdidn’t
thinktheconversionwouldbehardtoprogram.Itoccurredtome
thatthemoreverboseformatwouldbeagoodwaytoviewthe
informationin/etc/fstab,sothisscriptexiststoviewthefilein
thisformat.
HowItWorks
Sinceitwasdesignedforreformattingitsinput,awkisthelogical
tooltouse.Ifthelinebeginswithanoctothorpe(#),itisacomment,
anditisprintedunchanged.Otherlinesareprintedfieldbyfield,
withthenameofthefieldprecedingitsvalue.
Usage
showfstab[FILE...]
Ifnofileissuppliedonthecommandline,showfstabuses
/etc/fstab.
TheScript
awk'/#.*/{print;next}
{printf"device%s\n",$1
printf"\tmountpoint\t%s\n",$2
printf"\ttype\t%s\n",$3
printf"\toptions\t%s\n",$4
printf"\tdump\t%s\n",$5
printf"\tpass\t%s\n\n",$6
}'"${@:-/etc/fstab}"
Notes
Assumingthatthefieldsremaininthesameorderastheyappearin
/etc/fstab,theverboseformatcanberestoredtoitsoriginal
tersenesswith
awk'/#.*/{print;next}
NF{printf"%s\t",$2;next}
{print}
'
Ifthelinesmodifiedbyshowfstabarepipedthroughthis
snippet,theoutputissuitableforinclusioninetc/fstab:
/dev/hda1/ext3defaults11
/dev/hdb7/dataext3defaults12
16.14unique—RemoveAllDuplicateLines
fromaFile
AllUnixsystemshavetheuniqcommand,whichdeletes
consecutiveidenticallines,butthereisnostandardcommandto
removeduplicatelineswithoutfirstsortingthefile.
HowItWorks
Associativearraysinawkcanrememberpreviouslinesandcanbe
usedtosuppressprintingoflinesthathavealreadybeenseen.
Usage
unique[FILE...]
Oneormorefilesmaybesuppliedonthecommandline,andthe
standardinputwillbeusedifnofileisgiven:
$printf"%s\n"123245426|
unique
1
2
3
4
5
6
TheScript
Thepatternincrementsavariableaftertestingwhetherthearray
elementwhosekeyisthecurrentlineiszero.Ifitiszero,theline
hasnotbeenseenbefore,andthedefaultactionforlinesthatmatch
thepattern,print,isused.
awk'x[$0]++==0'${1+"$@"}
Summary
Thescriptsinthischapteraregenerallylesspolishedthanintherest
ofthebook,andsomeworkinlimitedenvironments.Morethanin
otherchapters,thecodehereisoftenmoreinterestingthanthescript
itself.Allthescriptsdosolveaproblemexpressedbysomeoneat
onetimeoranother,butsomeareincludedmostlybecausethe
techniquesusedcanbeappliedtosolvingotherproblems.
Footnotes
1 “TheArtofWritingLinuxUtilities,”http://www106.ibm.com/developerworks/library/l-util.html?ca
2 http://www.opengroup.org/onlinepubs/009695399/utilities/contents.html
Chapter17:ScriptDevelopment
Management
(1) Ontario,Canada
Theneedforascriptdevelopmentsystembecomesobviouswhen
youwanttomodifyscriptsthatareinuseonaregularbasisbyyou,
otherusers,orcronjobs.Youcan’taffordtohaveascriptfail;that
couldresultinanythingfromusersnotbeingabletoaccessthe
system,toaninvoicenotbeingsentout.
Ifliketheoriginalauthoryouarewritingscriptsforacoupleof
operatingsystems,adevelopmentmachineforeachisoutofthe
question(thoughyoucanhavetheoptionsofVM’sandDocker).
Thesolutionthathecameupwithwasusingthreedifferent
directories,onefordevelopingthescriptandonefordeployingand
oneforbackingupeachversionasanewonereplacedit.Thisisa
suggestiononly;therecouldbeothersolutionsthatmightworkfor
you.
Thescriptsunderdevelopmenthaveanextension,-shby
default,todistinguishthemfromproductionversions.Thebackups
arerenamedwithanumberedextensionthatisincrementedwith
eachsuccessivereplacement.Theextensioniszero-padded,with
threedigitsasthedefault.
Toaccessthescriptsfromtheterminal,youwouldalsoneeda
configurationfilethatspecifiesthedirectorywheretofindthe
scripts.Theconfigurationfileiscalledscript-setup.cfgandplaced
inthe$HOME/.configdirectoryastheotherconfigurationfileshave
beenplacedthroughoutthisbook.Thedirectoriesarecreatedbythe
script-setupscript.
17.1script-setup—PreparetheScripting
Environment
Tosetupthedirectoriesforascriptingsystem,theuserneedsthe
abilitytooverridethedefaultnames.Aplaywrightmightalready
haveadirectorycalledscriptsinhishomedirectory.Auser’sbin
directorymaybeladenwithanythingbutscripts(thoughifnoother
filesusethe-shextension,theremaybenoconflict).
Asetupscriptneedstopresentauserwiththedefaultdirectories
andfilesuffixes,andallowthemtobechangedaccordingtoone’s
wishes.
HowItWorks
Sinceauserisnotlikelytorunthisscriptmorethanonceortwice,
thisisastraightforwardtraditionalshellscriptwithnospecial
formatting.Itpresentstwomenusinsuccession,eachofferingthe
usertheopportunitytochangeeitherthedefaultsorthecurrent
settings.
Usage
script-setup
Thefirstmenupresentsthedirectories.Theusermaychange
anyorallofthembypressingtheappropriatenumber.
==============================================================
DirectoriesforScriptDevelopment
==============================================================
1.Scriptdevelopment:/home/cfaj/scripts
2.Installation:/home/cfaj/bin
3.Backup:/home/cfaj/scripts/bak
==============================================================
Select1to3tochangedirectoryor<ENTER>toaccept:
Thesecondmenupromptsforthesuffixesonthethreetypesof
file:development,production,andbackup.Theproductionscripts
havenosuffix,andthebackupsarepaddedwithzeroes:script-
setup-001,script-setup-002,andsoforth.
==============================================================
Suffixes
==============================================================
1.Developmentsuffix:-sh
2.Installedsuffix:
3.Backuppadding:3
==============================================================
Select1to3tochangesuffixor<ENTER>toaccept:
TheScript
Thecontentsof$defaultswillbewrittentotheuser’sscriptsetup.cfgfile.Byusingeval,variablesenteredbytheuserwillbe
expanded;inotherwords,ifuserenters /scripts,or
$HOME/scripts,then/home/user/scriptswillbeplacedinthefile.
set_defaults()
{
defaults="
##Directoryfordevelopmentcopyof
script
ScriptDir=${ScriptDir:-$HOME/scripts}
##Directoryforproductioncopyof
script
InstalDir=${InstalDir:-$HOME/bin}
##Directoryforoldversionsofscript
BackupDir=${BackupDir:$HOME/scripts/bak}
##Directoryforconfigurationfiles
ConfigDir=${ConfigDir:-$HOME/.config}
##Suffixfordevelopmentversionof
script
devel_suffix=-sh
##Suffixforproductionversionof
script(bydefault,none)
bin_suffix=
##Backupsuffixeswillbepaddedwith
zeroesto$VERSION_WIDTH
VERSION_WIDTH=3
##Verbosesetslevelofprogressand
errorreporting
verbose=0
##Copyrightstatement
copyright_blurb='
Thisisfreesoftware,releasedunderthetermsof
theGNUGeneral
PublicLicense.ThereisNOwarranty;notevenfor
MERCHANTABILITYor
FITNESSFORAPARTICULARPURPOSE.'
"
eval"$defaults"
}
write_config()
{
set_defaults##Expand
variables
printf"%s\n""$defaults">$configfile##Write
toconfigfile
}
Whenauserisaskedtoenteravalue,andwedon’twantany
previousvaluetobereplacedbyanemptystring,set_varreadsthe
inputintoadifferentvariableandonlyassignsittothedestinationif
itisnotempty.
set_var()
{
var=$1
prompt=$2
printf"\t%s:""$prompt"
read_var
[-n"$_var"]&&eval"$var=\"$_var\""
}
Theconfigurationfileisread,andthefourdirectoriesusedin
thischapter(orthedefaultvalues,ifthereisnoscript-setup.cfg
file)aredisplayed.Theusercanaccepttheseorenternewvalues.
Thedirectoriesarecreatediftheydonotalreadyexist.
set_dirs()
{
while:
do
set_defaults
dirx="Directoryexists"
##Checkwhetherthedirectoriesalreadyexist
[-d"$ScriptDir"]&&sde="[$dirx]"||sde=
[-d"$InstalDir"]&&bde="[$dirx]"||bde=
[-d"$BackupDir"]&&kde="[$dirx]"||kde=
##Printdirectoryinformation
printf"\n\n%s\n""$eq_bar"
printf"%s\n""DirectoriesforScript
Development"
printf"%s\n\n""$eq_bar"
w=-25##Displaywidthfordirectories
printf"1.Scriptdevelopment:%${w}s%s\n"
"$ScriptDir""${sde}"
printf"2.Installation:%${w}s%s\n"
"$InstalDir""${bde}"
printf"3.Backup:%${w}s%s\n"
"$BackupDir""${kde}"
printf"\n%s\n\n""$eq_bar"
printf"%s:""Select1to3tochangedirectory
or<ENTER>toaccept"
read_var
case$_varin
"")break;;
0|q)exit;;
1)set_varScriptDir"Scriptdevelopment
directory"
mkdir-p"$ScriptDir"||printf"\a"
;;
2)set_varInstalDir"Installation
directory"
mkdir-p"$InstalDir"||printf"\a"
;;
3)set_varBackupDir"Backupdirectory"
mkdir-p"$BackupDir"||printf"\a"
;;
esac
done
##CreatedirectoriesthatDon’tyetexist
fordirin$ScriptDir$InstalDir$BackupDir
do
[-d"$dir"]&&continue||mkdir-p"$dir"||
exit1
done
}
Theuserispromptedtoentersuffixestousefordevelopment
andproductionscripts,aswellastheamountofzero-paddingfor
backupversions.Bydefault,developmentscriptsuse-sh,and
productionscriptshavenosuffix.Thedefaultpaddingwidthis3.
set_suffixes()
{
while:
do
printf"\n\n%s\n""$eq_bar"
printf"%s\n""Suffixes"
printf"%s\n\n""$eq_bar"
printf"1.Developmentsuffix:%s\n"
"$devel_suffix"
printf"2.Installedsuffix:%s\n"
"$bin_suffix"
printf"3.Backuppadding:%s%s\n"
"$VERSION_WIDTH"
printf"\n%s\n\n""$eq_bar"
printf"%s:""Select1to3tochangesuffixor
<ENTER>toaccept"
read_var
echo
case$_varin
"")break;;
0|q)exit;;
1)set_vardevel_suffix"Developmentsuffix"
;;
2)set_varbin_suffix"Installedsuffix";;
3)set_varVERSION_WIDTH"Backuppadding";;
esac
done
}
progname=${0##*/}##Extractfilename
ofscriptfrom$0
##Decorativeline
eq_bar========================================================
configfile=$HOME/.config/script-setup.cfg##Configuration
file
set_defaults##Populatedefault
variables
##Loadconfigurationiffileexists
set_dirs##Getdirectorynames
fromuser
set_suffixes##Getsuffixesfrom
user
write_config##Writethesettings
totheconfigurationfile
printf"\n\n"##Keepthingstidy
(sincetherearenonewlinesafterprompts)
Notes
Itislefttotheusertoaddthescriptdevelopmentdirectorytothe
$PATHvariable.Thisshouldbedoneinwhicheverstartupfileyour
interactiveshellsources.Forbash,thiswouldbe$HOME/.bashrc.If
youusetheaddpathfunctionfromChapter7,thecommandtoinsert
isthis:
addpath$HOME/scripts
Adjustthepathifyoudidn’tusethedefaultdirectory.Ifyoudo
nothaveaddpathinstalled,addthefollowing(adjustingas
necessary):
PATH=$PATH:$HOME/scripts
17.2cpsh—InstallScriptandMakeBackup
Copy
Whenascriptisreadytobedeployed,theoldversion(ifany)needs
tobebackedupwithanincrementedsuffix,andthenewscriptmust
becopiedtotheproductiondirectorywithoutthedevelopment
suffix.
HowItWorks
Theconfigurationfile$HOME/.config/script-setup.cfgissourced
forthelocationsofthescriptdirectoriesandthesuffixesused.
Parameterexpansionisusedtoremoveanysuffix,ifnecessary,and
shellarithmeticandthe_zpadfunctionfromstandard-funcsin
Chapter1areusedtocreatetheincrementalbackupsuffix.
Usage
cpsh[-cCFG]SCRIPT...
Apartfromthe-coption,whichspecifiesadifferent
configurationfile,theonlycomplicationtousingthisscript,apart
fromsupplyingthecommandnames,isthatitmayneedtoberunas
rootifyoudonothavepermissiontowritetotheproduction
directory.Ifascriptistobedeployedto/usr/local/bin,for
example,youcouldgetanerrormessage:
$cpshcpsh
touch:cannottouch
`/usr/local/bin/cpsh':Permissiondenied
Toinstallinsuchalocation,youneedtobecomeroot,either
withsuorsudo.
TheScript
install_script()
{
filename=${file%$devel_suffix}##Command
name,withoutsuffix
dest=$InstalDir/$filename$bin_suffix##Pathto
installedfile
source=$ScriptDir/$filename$devel_suffix##Pathto
developmentfile
_uniqfile$BackupDir/$filename##
Incrementbackupfilename
bak=$_UNIQFILE##...and
storeas$bak
##Checkthatsourcefileexists
[-f"$source"]||return2
##Createdestinationfiletocheckpermissions(if
itdoesn’talreadyexist)
[-f"$dest"]||touch"$dest"||exit5
ifcmp"$source""$dest">/dev/null
then##Thefilesareidentical;donothing
echo"$sourceand$destarethesame">&2
else
##Copytheproductionfiletothebackup
directory
[-s"$dest"]&&cp-p"$dest""$bak"
##Copythedevelopmentscripttotheproduction
directory
cp-p"$source""$dest"
##Removewritepermissions
chmod+rx,-w"$dest"
fi
}
progname=${0##*/}
.standard-funcs##loadstandard
functions
##Usethescript-setupconfiguration
file
configfile=$HOME/.config/scriptsetup.cfg
##Iftheconfigurationfiledoesn’t
exist,runscript-setup
[-f$configfile]||script-setup||
exit5
##Sourceconfigurationfile
.$configfile
whilegetoptsc:arg;do
case$argin
c)configfile=$OPTARG
;;
*)exit1;;
esac
done
shift$(($OPTIND-1))
##Thisisonlynecessarywhenahandrolledconfigfileisused
checkdirs$HOME/.config$ScriptDir
$BinDir$BackupDir||
die$?"Couldnotcreate$dir"
##Installallcommandsgivenonthe
commandline
forscript
do
install_script"$script"
done
17.3shgrep—SearchScriptsforStringor
RegularExpression
Whenwritinganewfunction,youmightwanttocheckthatitwill
notcrashwithanypreviousfunction.Ifyouareeventhinkingof
deletingafunctionfromthelibrary,youneedtocheckifother
scriptsareusingit.Findoutifthescriptexistsandwhereitresides.
Forthesesituationsandothers,youwillrequireascriptthatwill
searchallyourscriptsforastringorregularexpression.
HowItWorks
Thescript-setup.cfgfileissourcedtogetthelocationofthe
scripts,andthesuffixusedonthedevelopmentversions.Byusing
thesuffix,anystrayfiles(backups,accidentalredirections,or
whatever)willbeignored.
Usage
shgrep[GREP_OPTIONS]REGEXP
Theshgrepscriptitselftakesnooptions,butwillpassallits
argumentstogrep;inthisway,youcanusewhateverfeaturesyour
versionofgrepsupports.
TheScript
##Usethescript-setupconfiguration
file
configfile=$HOME/.config/scriptsetup.cfg
##Iftheconfigfiledoesn’texist,
runscript-setup
[-f$configfile]||script-setup||
exit5
.$configfile
grep"$@"$ScriptDir/*$devel_suffix
17.4shcat—DisplayaShellScript
Ratherthanhavetohuntforthelocationofashellscript,youmight
liketohaveacommandthatwilldisplayitforyouwhenyoujust
providethename.
HowItWorks
Inbash,thetypecommandhasanoption,-p,thattellsitjusttoprint
thepathtothefilethatwouldbeexecuted,withoutanyother
verbiage.Inothershells,the$PATHvariableisexaminedwiththe
cmdpathfunction.Theresultofeithertypeorcmdpathisdisplayed
withtheuser’sdefault$PAGER,orless,ifthatvariableisnot
defined.
Usage
shcatSCRIPT
Ifthecommandenteredisnotascript,less(ifthatisyour
PAGER)willaskwhetheryoureallywanttoseeit:
$shcatls
"/bin/ls"maybeabinaryfile.Seeit
anyway?
Enteringnwillabortthecommand;ywilldisplayalotof
nonsense.
TheScript
cmdpath()
{
_CMD=${1##*/}
oldIFS=$IFS
IFS=:
set--$PATH
IFS=$oldIFS
for_DIR
do
if[-x"$_DIR/$_CMD"]
then
cmd=$_DIR/$_CMD
return
fi
done
}
if[-n"$BASH_VERSION"]
then
cmd=$(type-p"$1")
else
cmdpath"$1"
fi
${PAGER:-less}"$cmd"
Summary
There’snotagreatdealtothesuggestedsystemformanagingscript
development,butitisenoughtosavealotofworkandworry.The
mostimportantpartofitiscpsh,whichtakescareofinstallingand
backingupthescripts.Butisn’tsomethingmissing?Foralongtime
Ithoughtso:Whereisthescripttoreverttheinstalledcopytoan
earlierversion?
ForalongtimeIplannedtowritetherevershscript,butfinally
cametotheconclusionthatitreallywasn’tnecessary.Beingableto
runthedevelopmentversionmeansthatabadscripthardlyever
getsinstalled.I’veonlyhadtousethebackupstwoorthreetimesin
theyearsI’veusedthissystem.Inaddition,youcouldsimplyuse
GITfromthecommandlinetobackupyourscripts,ortorevertto
anearlierversion.
AppendixA:InternetScripting
Resources
(1) Ontario,Canada
AvastamountofinformationaboutshellscriptingisontheInternet.
Oneofthemostuseful,andwhereonecanlearnedagreatdeal,is
theUsenetnewsgroupcomp.unix.shell.Itsarchivescanbe
searchedathttp://groupsbeta.google.com/group/comp.unix.shell,butforposting,using
yourownnewsreaderisrecommendedovertheGoogleform.
There’snoshortageofnewsreaders,bothtextbasedandGUI:tin,
slrn,andpantonameafew,aswellascombinedmailand
newsreaders,suchaspineandMozillaThunderbird.
Listedhereareanumberofusefulwebsites.TheWWWis
notoriousforchanginglocations,andtheselinkswereallvalidat
thetimeofwriting;manyshouldbestableforalongtime.Chrishas
themallpostedonhiswebpageat
http://cfaj.freeshell.org/shell,whereheshalltrytokeepthem
currentandaddnewpages.
IntroductionstoShellScripting
http://www.linuxcommand.org/writing_shell_scripts.php:
Writingshellscripts,byWilliamShotts,Jr.
http://steve-parker.org/sh/sh.shtml:SteveParker’s
Bourne/Bashshellscriptingtutorial.
http://www-128.ibm.com/developerworks/linux/library/lbash.html:Bashbyexample:Fundamentalprogrammingin
theBourneagainshell(bash),byDanielRobbins.
http://www.ibiblio.org/mdw/HOWTO/Bash-Prog-IntroHOWTO.html:BASHProgramming-IntroductionHOW-TO,by
MikeG.
http://www.icon.co.za/ psheer/book/node10.html.gz:Rute
User’sTutorialandExposition:ShellScripts,byPaulSheer.
IntermediateandAdvancedScripting
http://www.grymoire.com/Unix/:BruceBarnett’stutorialson
UNIXshellprogramming.
http://www.tldp.org/LDP/abs/html/index.html:Advanced
Bash-ScriptingGuide,byMendelCooper.
http://home.comcast.net/ j.p.h/cus-faq.html:The
comp.unix.shellFAQ,compiledbyJoeHalpin.
http://code.dogmap.org/lintsh/:PaulJarc’sshellconstructs
page.
CollectionsofScripts
http://www.shelldorado.com:Heiner’sSHELLdorado;links,
tutorials,andalotofscripts.
http://www.mtxia.com/fancyIndex/Tools/Scripts/Korn/:
DanaFrench’sKornShellscripts.
HomePagesforShells
http://cnswww.cns.cwru.edu/ chet/bash/bashtop.html:Chet
Ramey’sBashhomepage.
http://kornshell.com:TheKornShellhomepage.
http://www.cs.mun.ca/ michael/pdksh:pdksh—thePublic
DomainKornShell.
RegularExpressions,sed,andawk
http://www.linuxfocus.org/English/July1998/article53.html
:RegularExpressions,byGuidoSocher.
http://etext.lib.virginia.edu/services/helpsheets/unix/reg
:UsingRegularExpressions,byStephenRamsay.
http://sitescooper.org/tao_regexps.html:ATaoofRegular
Expressions,bySteveMansour.
http://www.amk.ca/python/howto/regex:RegularExpression
HOWTO,byA.M.Kuchling.ThisisPython-specific,butthe
principlesapplytootherlanguages,aswell.
http://www.faqs.org/faqs/computer-lang/awk/faq:Theawk
FAQ,maintainedbyRussellSchulz.
http://www.student.northpark.edu/pemente/sed:Eric
Piment’ssedpage.
MiscellaneousPages
http://www.gnu.org/software/textutils/manual/textutils/htm
:Openingthesoftwaretoolbox,byArnoldRobbins.
http://www-128.ibm.com/developerworks/linux/library/lutil.html:TheartofwritingLinuxutilities,byPeterSeebach.
http://www.gnu.org/software/autoconf/manual/autoconf-
2.57/html_node/autoconf_114.html#SEC114:PortableShell
Programming.
HistoryoftheShell
http://www.cnop.net/staticpages/index.php/intro-unix:
StevenBourne’sAnIntroductiontotheUnixShell.A
descriptionoftheoriginalBourneshell,byitsauthor.
http://www.in-ulm.de/ mascheck/bourne:TheTraditional
BourneShellFamily,bySvenMaschek.
http://www.faqs.org/faqs/unix-faq/faq/part3/section16.html:Whydosomescriptsstartwith#!…?
http://groupsbeta.google.com/group/alt.folklore.computers/msg/a176a30b5
:HistoryofUnixshells,byJohnMashey.
http://groups.google.com/groups?hl=en&lr=&ie=UTF-
8&selm=1994Feb23.235836.15874%40sq.sq.com:Unixbeforesh,
byMarkBrader.
Index
A
addpathfunction
Aliases
AmericanStandardCodeforInformationInterchange(ASCII)
B
basenamefunction
Bookkeepingsystem
gle
cleanup()
computerizedaccountingsystem
entrydate
get_form()
gle_init()
importtospreadsheet
incomeandexpensesstatement
is_changed()
ledgerfilefields
lookup()
POSIXshell
query_save()
save_transaction()
shoebox
sh_read()
sort_amounts()
standard-funcs
transactionrecords
tr_field
tr_form
tr_init()
prcalc
arithmeticoperator
desktopprintingcalculator
initializationofvariables
operation
sh_readfunction
STDERR
Bourneshellexpansions
C
cgi-funcslibrary
dehex
filedate
twodigithexadecimalnumbertodecimal
Characterfunction
ascfunction
decimalnumber
lowercase
nxtfunction
uppercase
checkpathfunction
Commandlinearguments
functions
abbrev_num
arg
checkdirs
checkfiles
cleanup
commandsubstitution
commas
date_vars
die
faster
get_key
getline
is_num
$menuvariable
press_any_key
printarguments,
show_date
zpad
shellcommands
See (Shellcommands)
specialparameters
Unixutilities
awk
cat
date
file
grep
Is
PSandPDFfiles
sed
split
sudo
tr
uniq
wc
which
Comma-separatedvalues(CSVs)
CommonGatewayInterface(CGI)programs
Convertingtextfiles
MacintoshfilestoWindows
MacintoshfilestoUnix
Unixfiletowindows
UnixfilestoMacformat
Windowsfilestomacintosh
windowstoUnix
_cubefunction
D
Databases
lookupscript
PhoneBase
ph
phadd
phdel
phx
shdb-funcs
assemblingCSV
csv_split
db-demo
load_db
put_record
split_record
TorontoFree-Netmembershipdatabasescreen
Datefunctions
date2julianfunction
dateshiftfunction
day_of_weekfunction
days_in_monthfunction
diffdatefunction
display_datefunction
is_leap_yearfunction
parse_datefunction
split_datefunction
valid_datefunction
dateshiftfunction
day_of_weekfunction
days_in_monthfunction
diffdatefunction
dirnamefunction
display_datefunction
Displayingfiles
prn
prw
wbl
E
ExtendedBinary-CodedDecimalInterchangeCode(EBCDIC)
F,G
File-agingsystem
date-file
keepnewest
rmold
Filenamefunctions
basenamefunction
dirnamefunction
fixfnamefunction
fix_pwdfunction
is_dirfunction
is_OKfnamefunction
is_pfnamefunction
is_whitespacefunction
new_filenamefunction
nodoublecharfunction
OKfnamefunction
pfnamefunction
whitespcfunction
fixfnamefunction
fix_pwdfunction
flush_rightfunction
Formattingfile
fed
finfo
lfreq
wfreq
_fpmulfunction
H
Housekeeping
dfcmp
lsr
sym2file
symfix
undup
zrm
HTMLfiles
CGIscript
mk-htmlindex,creation
pretext
text2html
html-funcslibrary
get_element
html-title
split_tags
I,J,K
Internetscriptingresources
collections
historyshell
homepages
intermediateandadvancedscripting
miscellaneouspages
regularexpressions
shellscripting
_intfunction
is_dirfunction
is_leap_yearfunction
is_OKfnamefunction
is_pfnamefunction
is_whitespacefunction
L
Linenumbers
M
Mathfunctions
_cube
_fpmul
_int
mean
median
mode
_pow
range
_round
_square
stdev
_total
unitconversion
centimeterstoinches
degreescelsiustofahrenheit
degreesfahrenheittocelsius
degreesfahrenheittoKelvins
gramstoounces
inchestocentimeters
inchestomillimeters
Kelvinstodegreesfahrenheit
kilogramstopounds
kilometerstomiles
menusystem
meterstoyards
milestokilometers
millimeterstoinches
ouncestograms
poundstokilograms
yardstometers
meanfunction
modefunction
N
new_filenamefunction
nodoublecharfunction
O
OKfnamefunction
P,Q
parse_datefunction
PATHvariable
addpathfunction
checkpathfunction
definition
pathcommand
rmpathfunction
unslashfunction
pfnamefunction
POSIXparameterexpansions
POSIXshell
aliases
arithmeticexpressions
bourneshellexpansions
filesource
parameterexpansions
pathnameexpansion
regularexpressions
shell-specificexpansions
_powfunction
pseudo-randomnumbergenerator(PRNG)
R
randomfunction
dice
PRNG
rand-date
$RANDOMvariable
randomword
randsort
randstr
throwing
tosssimulation
rangefunction
rmpathfunction
_roundfunction
S
Screen-funcslibrary
box_area,box_area_at
box_block,box_block_at
center,string
clear_area,clear_area_at
clear
cursorposition
flush_rightfunction
horizontalbarprinting
get_size,LINESandCOLUMNSvariables
max_length
print_block_at
put_block_at
ruler
screen-demo
screen-vars
set_attrvariable
set_fg,set_bg,set_fgbg
standard-funcs
verticalbarprinting
Screenmanipulation
Scriptdevelopmentsystem
addpathfunction
cpsh
directories
script-setup.cfgfile
set_dirs()
set_suffixes()
set_var()
shcat
shgrep
suffixes
Shellcommands
case
echo
eval
getopts
local
printf
set
shift
type
Smorgasbordscripts
cci
cwbw
flocate
intersperse
ipaddr
ipaddr.cgi
iprev
llscript
name-split
one-timescript
POSIXmanpage
rot13
showfstab
topntail
uniqcommand
split_datefunction
_squarefunction
stdevfunction
String
char-funcsLibrary
ASCIIsequence
convertingdecimalequivalent
decimalnumber
lowercase
uppercase
substrfunction
gsubfunction
index/rindexfunctions
insert_str
repeatfunction
replacing
T
_totalfunction
U
UKAdvancedCrypticsDictionary(UKACDfile)
Unixutilities
awk
cat
date
file
grep
Is
PSandPDFfiles
sed
split
sudo
tr
uniq
wc
which
unslashfunction
V
valid_datefunction
W,X
whitespcfunction
WordFinderfunction
carriagereturnsanddeletes
configurationfile
dictionarydirectory
matchingwords
aminus
anagramscript
aplus
beginningpattern(wfb)
containingpattern(wfc)
endingpattern(wfe)
wfit
sorting
squishcompound
webpage
wf-setupscript
Y,Z
Year2000(Y2K)bug

The Script - Kitabsiz.az

Transcription

Similar documents

Tweaks for better Performance

Audio description

Flemish String Board Attachment

Ford Oval and Mustang® are registered trademarks owned and

Preoperative Measurements and Intraocular Lens

Site Report Poroton Home

Project: Divisidero Shell Gas Station, San Francisco, CA

Auto Mouse Click by MurGee.com - MurGee Auto Mouse Click Video

SEPT. 20 2006

Rules for All Participants