Topic: "fork" in a CGI script results in a hanging web page
Dear "pythoners",
until about two years ago I wrote my scripts in Perl, then changed to Python. I tried to translate one of my previously nicely working Perl scripts into Python, but it fails to work in a proper way. I'm at my wits end and would like to ask you for help. The problem I'm trying to solve is connected to the "fork" command and its properties with respect to STDERR, STDIN and STDOUT on OS level, and I'm lacking knowledge in this field.
The basic idea of the CGI script is this: sometimes I have to create web services which deal with huge amounts of data and thus have long processing times. In order not to have a frustrated user sitting in front of an inactive screen, asking himself whether the script has crashed, I'm creating a "Patience, please" page, which is constantly refreshed, displaying a time count and status messages. This page is switched for a results page as soon as the results have been calculated.
This could have been realized through a series of scripts, but to my point of view a more elegant solution is to have everything in just one script. This can be done by setting up a web form which collects all required input. When the user submits the input the script calls itself - with altered parameter set - "forks" the process, calculates the requested data in one branch of the "fork" and produces (and refreshes) the "Patience" page in the other "fork" branch, until the data production is completed. When that status is achieved the "fork" is terminated and the browser jumps to the results page.
I already had a problem in the Perl version, which I only could solve with the help of Perl forum members. Now again I haven't been successful: seemingly the branches of a "fork" aren't entirely independend, the standard I/O has to be closed, otherwise the "Patience" page is hanging, until the data processing is done, the counts for a second, just to immediately switch to the "Results" page. However, if all I/O channels are closed and re-opened (otherwise the HTML page couldn't be created), the data creation hangs.
I tried quite a number of things, including the replacement of the "fork" through a "popen", etc, but I couldn't find a workaround. Who can help me? What is wrong with the script, and how can I get the script running?
Here is the commented source code for a little test script:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os, re, sys, cgi, time
import cgitb; cgitb.enable()
Change = "Dec-24-2011"
# ### START SUBROUTINE SECTION ############################################################################ #
def ErrorMsg(Stat,Mess): #** Creates an "Error" message; ***************************************************#
Html = "Content-Type: text/html\n\n"
Html += "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n\n<html>\n"
Html += "<head>\n <title>DEMO Error Message</title>\n"
Html += " <!-- Last changes: %s -->\n</head>\n\n" % (Change)
Html += "<body>\n<center>\n\n"
Html += "<span style=\"font-size:22pt; color:Green\">DEMO ERROR MESSAGE</span><br>\n<br>\n"
if (Stat == "0"):
Html += "!!! System error - could NOT fork process !!!<br>\n"
elif (Stat == "1"):
Html += "!!! System error - could NOT write \"HTML\" file '%s'!!!<br>\n" % (Mess)
Html += "</center>\n</body>\n</html>\n"
print Html
sys.exit(0);
def StartPage(): #** Creates the "Start Page"; *************************************************************#
Html = "Content-Type: text/html\n\n"
Html += "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n\n<html>\n"
Html += "<head>\n <title>DEMO START PAGE</title>\n"
Html += " <!-- Last changes: %s -->\n</head>\n\n" % (Change)
Html += "<body>\n<center>\n<br>\n"
Html += "<span style=\"font-size:22pt; color:Green\">DEMO START PAGE</span><br>\n<br>\n"
Html += "<form action=\"%s\" method=\"post\">\n" % (CGIPath)
Html += "<input type=\"hidden\" name=\"STAT\" value=\"1\">\n"
Html += "<b>TYPE A "TICK NUMBER":</b><br>\n"
Html += "<input type=\"text\" name=\"Tick\" size=\"10\"><br><br>\n"
Html += "<input type=\"submit\" value=\"PROCESS\"> <input type=\"reset\" value=\"RESET FORM\">\n"
Html += "</form>\n</center>\n</body>\n</html>\n"
return Html
def ProcessPage(): #** Creates the "Process Page"; *********************************************************#
# #
# If both STDERR calls are run, after sending the form page the program switches to the waiting page and #
# counts seconds - without switching to the "Results Page" #
# #
# The same happens if only the first STDERR call is run #
# #
# If just the second STDERR call is run after sending the form page the program switches to the initial #
# "Waiting Page", then stays there until the long process is finished, and AFTER that switches to the #
# "Waiting Page", counting seconds - without switching to the "Results Page" #
# #
# If none of them is called after sending the form page the program switches to the initial "Waiting Page", #
# then stays there until the long process is finished, and then switches to the "Results Page". This is #
# what is intended, except for that between the switch to the initial "Waiting Page" and the final switch #
# to the "Results Page" seconds should have been counted #
# #
# None of the STDERR calls makes it into the "error.log" - which doesn't surprise, because STDERR is #
# obviously closed #
# #
## sys.stderr.write("%s: CHILD: PROCESS PAGE started\n" % (CGIPath)) ### First STDERR call ###
Html = "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n<html>\n"
Html += "<head>\n <title>DEMO PROCESS PAGE</title>\n"
Html += " <meta HTTP-EQUIV=\"Expires\" CONTENT=\"NOW\">\n</head>\n\n"
Html += "<body>\n<center>\n<br>\n"
Html += "<span style=\"font-size:22pt; color:Green\">DEMO PROCESS PAGE</span><br>\n<br>\n"
Html += "<span style=\"font-size:22pt; color:Violet\">...THAT'S THE TEST...</span><br>\n<br>\n"
Html += "...<a href=\"%s\">go back to the DEMO start page</a>...<br>\n" % (CGIPath)
Html += "</center>\n</body>\n</html>\n"
#---- >>>>> ------------------------------------------------------------------------------------------------#
time.sleep(float(Ticks)) # Simulation of a long process #
#---- <<<<< ------------------------------------------------------------------------------------------------#
## sys.stderr.write("%s: CHILD: PROCESS PAGE created\n" % (CGIPath)) ### Second STDERR call ###
return Html
def WaitingPage(Url,Tact): #** Creates the "Waiting Page"; *************************************************#
Html = "Content-Type: text/html\n\n"
Html += "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 Transitional//EN\">\n<html>\n"
Html += "<head>\n <title>...DEMO is running...</title>\n"
Html += " <META HTTP-EQUIV=\"refresh\" content=\"%s; url=%s\">\n</head>\n\n" % (Tact,Url)
Html += "<body>\n<center>\n<br>\n"
Html += "<span style=\"font-size:22pt; color:Red\">...DEMO is running...</span><br>\n"
Html += "<br>\n<span style=\"font-size:22pt\">Don't go BACK!</span><br>\n<br>\n"
Html += "<span style=\"font-size:20pt; color:Blue\">Elapsed Time: %s</span><br>\n" % (Elapsed)
Html += "</center>\n</body>\n</html>\n"
return Html
def CurrTime(): #** Creating a string with the current time; ***********************************************#
Time = time.time()
(Yr,Mo,Da,Hr,Mi,Sc,WD,DY,ST) = time.localtime(Time)
LocTime = '%02d:%02d:%02d' % (Hr,Mi,Sc)
return LocTime
def ElapsedTime(): #** Calculates the elapsed time *********************************************************#
Time = time.time()
(Yr,Mo,Da,Hr,Mi,Sc,WD,DY,ST) = time.localtime(Time)
CTime = (((Hr * 60) + Mi) * 60) + Sc
Digits = STime.split(":")
Hr = int(Digits[0])
Mi = int(Digits[1])
Sc = int(Digits[2])
ITime = (((Hr * 60) + Mi) * 60) + Sc
if (CTime < ITime):
CTime += 86400
ETime = CTime - ITime
Sc = ETime % 60
ETime = (ETime - Sc) / 60
Mi = ETime % 60
Hr = (ETime - Mi) / 60
ETime = "%02d:%02d:%02d" % (Hr,Mi,Sc)
return ETime
# ### END SUBROUTINE SECTION ############################################################################## #
# ### START MAIN PROGRAM ################################################################################## #
CGIPath = os.environ['SCRIPT_NAME']
TmpAbs = "/var/www/tmp" # Absolute 'tmp' directory path #
TmpVrt = "/tmp" # Virtual 'tmp' directory path #
HtmOut = "%s/demo.html" % (TmpAbs) # HTML Output File #
HtmTmp = "%s/demo.html.tmp" % (TmpAbs) # "Temporary" HTML Output File #
Form = cgi.FieldStorage()
STime = Form.getvalue('Time',"00:00:00") # "Process Start Time" #
Ticks = Form.getvalue('Tick',0) # "Seconds to wait" #
Status = Form.getvalue('STAT',"0") # "Process Status" #
sys.stdout = os.fdopen(sys.stdout.fileno(),'w',0)
Elapsed = "00:00:00" # Initialize "Elapsed Time" #
##### Status = 0: Create the "Start Page" ###################################################################
if (Status == "0"):
print StartPage()
##### Status = 1: Fork the program, run the long process and create the "Results Page" in CHILD #############
##### and an "Initial Waiting Page" in PARENT ###################################################
elif (Status == "1"):
if ((os.path.exists(HtmOut)) and (os.path.getsize(HtmOut) > 0)):
os.unlink(HtmOut)
try:
HTMTMP = open(HtmTmp,"w")
except IOError:
ErrorMsg("1",HtmTmp)
try:
ForkVal = os.fork()
except IOError:
ErrorMsg("0","")
if (ForkVal == 0): # CHILD process #
# If STDOUT, STDERR and STDIN are NOT closed the program remains in the "Start Page" until the long #
# process is finished, then switches to the "Waiting Page" and immediately switches to the "Results Page" #
sys.stdout.close() # Close STDOUT #
sys.stderr.close() # Close STDERR #
sys.stdin.close() # Close STDIN #
# Although STDERR is closed the attempt to write into STDERR in 'ProcessPage' function influences the #
# program behaviour drastically (look into the source code above) #
Html = ProcessPage()
HTMTMP.write(Html)
HTMTMP.close()
os.rename(HtmTmp,HtmOut)
else: # PARENT process #
Elapsed = "00:00:00"
Url = "%s?STAT=2&Time=%s" % (CGIPath,CurrTime())
print WaitingPage(Url,1)
##### Status = 2: Create the "Waiting Page" #################################################################
elif (Status == "2"):
Elapsed = ElapsedTime()
if ((os.path.exists(HtmOut)) and (os.path.getsize(HtmOut) > 0)):
Url = "%s/demo.html" % (TmpVrt)
print WaitingPage(Url,0)
else:
Url = "%s?STAT=2&Time=%s" % (CGIPath,STime)
print WaitingPage(Url,1)
sys.exit(0)
# ### END MAIN PROGRAM #################################################################################### #Who can help me?
Many thanks in advance!
With all of my best greetings and wishes,
Yours
chanklaus
Last edited by chanklaus (December 16, 2011 13:41)
