Previous Thread
Next Thread
Print Thread
Hop To
#179918 03/03/2007 10:20 AM
Joined: Jun 2006
Posts: 136
O
member
member
O Offline
Joined: Jun 2006
Posts: 136
i have written some php code -- about 400 lines -- that allows me to pull data off particular websites (nothing too naughty; it's public financial data from the central bank).

the php code takes up to 60 seconds to finish execution, mainly because it pulls data down from about 4 rather slow sites.

the code finally produces a small HTML file comprising collected data that is imported and displayed in a side bar custom island.

my issue is this: i wish to run this code about every 30 minutes to keep my site up to date with local financial data. where should i place this code? can i call it from the body section of the custom islands? and if i do, will there be any speed consequences for my users?

i'd be thankful for anyone's thoughts on this.





Okusi Associates -- Indonesian business and management services
Joined: Jun 2006
Posts: 16,299
Likes: 116
UBB.threads Developer
UBB.threads Developer
Joined: Jun 2006
Posts: 16,299
Likes: 116
You could start running into the php max execution limit if it takes that long to run... Also, I'd think of doing a cronjob to pull the data vs loading it in a content island, as the lag it takes to generate the data could hold the forum from loading...

I'd be interested at looking at your code though wink


I am a Web Development Contractor, I do not work for UBBCentral. I have provided free User to User Support since the beginning of these support forums.
Do you need Forum Install or Upgrade Services?
Forums: A Gardeners Forum, Scouters World
UBB.threads: UBBWiki, UBB Styles, UBB.Sitemaps
Longtime Supporter & Resident Post-A-Holic
VNC Web Services: Code Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!
Joined: Jun 2006
Posts: 136
O
member
member
O Offline
Joined: Jun 2006
Posts: 136
Originally Posted by Gizmo
You could start running into the php max execution limit if it takes that long to run...

yes, i've already come across the problem of timeout. most systems are set to 30 seconds. i used ini_set("max_execution_time", "60") to fix it.

Quote
Also, I'd think of doing a cronjob to pull the data vs loading it in a content island, as the lag it takes to generate the data could hold the forum from loading...

you mean, call php from the command line, called from cronjob?

php < code.php

i can't seem to bring php up like this ... (unix is not really my forte).

Quote
I'd be interested at looking at your code though wink

well, if you show me yours, i'll show you mine wink

actually, it's nothing too special. just a lot of data munging to extract data from particular locations within tables.

and a few data cleanup routines to put the information in a useable format.

i'm not sure it would be of generic interest.


Okusi Associates -- Indonesian business and management services
Joined: Jun 2006
Posts: 136
O
member
member
O Offline
Joined: Jun 2006
Posts: 136
ok, i can now run phh from the unix command line! and the program executes successfully like this. any advice on how can i set up a cron job to run this every 30 minutes?? last time i set up a cron job was in 1993, and i seem to recall some angry sysops ... smile


Okusi Associates -- Indonesian business and management services
Joined: Jun 2006
Posts: 136
O
member
member
O Offline
Joined: Jun 2006
Posts: 136
another problem ... it seems my ISP has not given me permission to use crontab. maybe they heard about the last time i used it.

so, seems i could be back to having to execute the php code from within ubb.

would embedding an exec() call in the PHP section of the custom island set up be a problem?


Okusi Associates -- Indonesian business and management services
Joined: Jun 2006
Posts: 136
O
member
member
O Offline
Joined: Jun 2006
Posts: 136
the exec() didn't work out. so i inserted

require_once("http://jakchat.com/code.php");

at the top of the PHP section in the custom island concerned -- which i presume imported all 400 lines of code, with all its functions -- and this seems to work. i have noticed no speed hits when using the system, using an update time of 4 minutes for testing purposes.

nonetheless, makes me a bit nervous putting so much complex code in that section ...


Okusi Associates -- Indonesian business and management services
Joined: Jun 2006
Posts: 16,299
Likes: 116
UBB.threads Developer
UBB.threads Developer
Joined: Jun 2006
Posts: 16,299
Likes: 116
Originally Posted by KuKuKaChu
yes, i've already come across the problem of timeout. most systems are set to 30 seconds. i used ini_set("max_execution_time", "60") to fix it.
Still not really a good answer, perhaps splitting it up into several seperate scripts would be better; if it's taking that long to run, that's way too long imo...

Originally Posted by KuKuKaChu
you mean, call php from the command line, called from cronjob?

php < code.php

i can't seem to bring php up like this ... (unix is not really my forte).
No, more setting a cron task to execute: php /path/to/code.php

Originally Posted by KuKuKaChu
well, if you show me yours, i'll show you mine wink

actually, it's nothing too special. just a lot of data munging to extract data from particular locations within tables.

and a few data cleanup routines to put the information in a useable format.

i'm not sure it would be of generic interest.


Which one did you want to see? lol...

As for using exec or system, some isp's block them out, if you where to execute them from command line you could have them run fine in the back end of things, cron would execute them on a set schedual and you wouldn't have to worry about any potention forum lag.

BTW, you may not notice any lag now, but trust me, you will lol...


I am a Web Development Contractor, I do not work for UBBCentral. I have provided free User to User Support since the beginning of these support forums.
Do you need Forum Install or Upgrade Services?
Forums: A Gardeners Forum, Scouters World
UBB.threads: UBBWiki, UBB Styles, UBB.Sitemaps
Longtime Supporter & Resident Post-A-Holic
VNC Web Services: Code Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!

Link Copied to Clipboard
ShoutChat
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Recent Topics
Bots
by Outdoorking - 04/13/2024 5:08 PM
Can you add html to language files?
by Baldeagle - 04/07/2024 2:41 PM
Do I need to rebuild my database?
by Baldeagle - 04/07/2024 2:58 AM
This is not a bug, but a suggestion
by Baldeagle - 04/05/2024 11:25 PM
Is UBB.threads still going?
by Aaron101 - 04/01/2022 8:18 AM
Who's Online Now
0 members (), 764 guests, and 261 robots.
Key: Admin, Global Mod, Mod
Random Gallery Image
Latest Gallery Images
Los Angeles
Los Angeles
by isaac, August 6
3D Creations
3D Creations
by JAISP, December 30
Artistic structures
Artistic structures
by isaac, August 29
Stones
Stones
by isaac, August 19
Powered by UBB.threads™ PHP Forum Software 8.0.0
(Preview build 20230217)