Previous Thread
Next Thread
Print Thread
Hop To
7.6.2 file attachment cleanup #262011 01/09/2019 8:29 AM
Joined: Jun 2006
Posts: 955
Zarzal Offline OP
Old Hand
OP Offline
Old Hand
Joined: Jun 2006
Posts: 955
I run my ubb forum for over 20 years and do several updates and upgrades. In the past there must something goes wrong with my attachments.

The control panel tells me that I have 867 files. In my server filesystem I can find 743 files. I knew the table ubbt_FILES which contains 1194 rows (this counts attachments and gallery pictures together). I found postings marked with attachment and the file is gone, I found files without any relations to a post and so on. I see the difference in handling attachments in the server directory. New attachments have only a number, old attachments have a name that the attachment shows up in the posting. So I can see what is new and what is old in the filesystem.

Now I go to cleanup it a litte bit. I check ubbt_FILES and try to find the post or file which belongs to a row. The table ubbt_FILES contans a field 'POST_ID'. Is it the propper way to find the post while entering this url?
http://<path to board>/ubbthreads.php/topics/<POST_ID>

Sometime I found a thread which contains the post, sometimes I got the message that nothing can be found. I'm not sure. Is this the right way to call a single post in the webbrowser?
My ubbt_FILES table has some strange entrys with FILE_SIZE = NULL and FILE_DIR = NULL and if I have some time I try the POST_ID from such a strange row. Is it save to delete the row If my forum tell me that the posting no longer exists?

Regards
Zarzal


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Express Hosting
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262013 01/09/2019 11:33 AM
Joined: Dec 2003
Posts: 5,891
Ruben Offline
Offline
Joined: Dec 2003
Posts: 5,891
This has plagued ubb in the past, In the early days
The one case where you can't find a post is probably due to deleted posts and the attachment becomes orphaned.
(not deleted) Either in the table and or file folder.
In the case where you do find a post missing an attachment could be any scenario.
Such as moving hosts and a file did not get transferred or the file got corrupted ,etc.

In past versions Rick never could come up with a method to clean them all up.
It probably should be added as a utility in the rebuilders section of the control panel.
But most people that find the issue are people that have used UBB over many versions and a couple of hosts.
Like yourself.
So probably current versions don't have this problematic issue as drastic as the very old versions.


I am sure that isaac and Gizmo are aware of the issue. But with all the coding cleanup they have been working on plus getting UBB compatible with php7 ,they probably have not revisited the issue to see if it can be addressed yet.
Their laundry list is probably a lot larger than ours.

As far as posts with missing attachment files go..
For now you could just manually edit the post and delete the code in the post for the attachment if in fact the image does not exist. Till you hear from the programmers.
If you just delete the bad row in the ubb_files table. The post body and post default body will still have a reference to the attachment.


Last edited by Ruben; 01/09/2019 11:46 AM. Reason: added comment

Blue Man Group
There is no such thing as stupid questions. Just stupid answers
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262015 01/09/2019 12:42 PM
Joined: Jun 2006
Posts: 955
Zarzal Offline OP
Old Hand
OP Offline
Old Hand
Joined: Jun 2006
Posts: 955
Yes, I can confirm all of it.
Thanks for pointing me to both references. Better I do nothing directly in the table.
In the past we had somewhere a script call 'scan for orphaned files' but I think this was for V6.x. It's to long time ago and I can't remember anymore.Maybe on UBBDev.

Its hard to find postings with attachments if you have >50 forums with 107471 postings. There is no search option to find postings with attachments. This would help.
So we note: If V7.7 released and php7 ready the file managmend need a little rework smile


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Re: 7.6.2 file attachment cleanup [Re: Ruben] #262016 01/09/2019 12:42 PM
Joined: Apr 2004
Posts: 1,405
isaac Online Splat
UBB.threads Developer
Online Splat
UBB.threads Developer
Joined: Apr 2004
Posts: 1,405
Originally Posted by Ruben
I am sure that isaac and Gizmo are aware of the issue. But with all the coding cleanup they have been working on plus getting UBB compatible with php7 ,they probably have not revisited the issue to see if it can be addressed yet.

Thanks Ruben thumbsup

Around July 2018, a lot of work had gone in to getting mature forums with an crazy amount of attachment files sorted out. The feature that Zarzal had "discovered" in UBBT 7.6.2 was a small part of that work. Gizmo has created three different tools for the job. I've modified one of those tools to further his work, for performing a related job. It really is a *job* though.

1) UBBT Orphand File Fix - deals with the database and removes the missing data items.
2) UBBT Attachments - deals with the files and moves the additional files.
3) UBBT Attachments DB - updates the database with the correct file sizes.
and...
4) UBBT Attachments Folder - populate the file sizes and image-dimensions of pre-UBB.threads 7.5 attached files.

I don't recall if those are the correct names. The tools were written over 6 months ago. Lots has happened since then.

For most people, these tools are unnecessary. For every one, the job is long and complex to perform. The tools may never be made public. As I also recall, these tools were basically created for "contracted work" or "as a favor." They may only become used as one of the many utilities that keep Gizmo on top of his game for being really good at what he does. If this sounds like a task you feel your forums need sorting, then you need to contact him and negotiate a price for either him to do the job, or for purchasing the tools from him -- but extreme warning, they do require deep knowledge of WTF you are doing, as they may break your whole forum and database if executed without any education of what step you're trying to do.


isaac @ id242.com // my forum @ CelicaHobby.com
a current developer of UBB.threads php forum software // 7.7.2 Progress Notes
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262017 01/09/2019 1:34 PM
Joined: Dec 2003
Posts: 5,891
Ruben Offline
Offline
Joined: Dec 2003
Posts: 5,891
Originally Posted by Zarzal
Yes, I can confirm all of it.
Thanks for pointing me to both references. Better I do nothing directly in the table.
In the past we had somewhere a script call 'scan for orphaned files' but I think this was for V6.x. It's to long time ago and I can't remember anymore.Maybe on UBBDev.

Its hard to find postings with attachments if you have >50 forums with 107471 postings. There is no search option to find postings with attachments. This would help.
So we note: If V7.7 released and php7 ready the file managmend need a little rework smile

I know all to well how cumbersome this can be.
I dealt with one site that was a photography site..
The problem was they had like over 40,000 image attachments.
But the server would only let me see 10,000 in filezilla. and cp filemanger was very slow to the point
I finally gave up on it..


Blue Man Group
There is no such thing as stupid questions. Just stupid answers
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262018 01/09/2019 2:05 PM
Joined: Apr 2004
Posts: 1,405
isaac Online Splat
UBB.threads Developer
Online Splat
UBB.threads Developer
Joined: Apr 2004
Posts: 1,405
10,000 files limitation is a ftp configuration setting. You can set it to higher or lower if you change the setting in your ftp software and/or the server. The FileZilla program states in its support forums that it has no limitation. Meaning, the issue you had was most likely the server was set (usually at a default limit) to truncate the directory listing for.... directory listing speed. You still have access to all the files in the directory. And the server has zero issues dealing with them. In this case, SSH is your friend to access the server via SSH CLI.


isaac @ id242.com // my forum @ CelicaHobby.com
a current developer of UBB.threads php forum software // 7.7.2 Progress Notes
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262021 01/09/2019 2:20 PM
Joined: Dec 2003
Posts: 5,891
Ruben Offline
Offline
Joined: Dec 2003
Posts: 5,891
Yes I know it was a server issue and no access to increase it . It was a simple cheapo shared hosting account. and the host was not willing to change it.


Blue Man Group
There is no such thing as stupid questions. Just stupid answers
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262022 01/09/2019 2:35 PM
Joined: Apr 2004
Posts: 1,405
isaac Online Splat
UBB.threads Developer
Online Splat
UBB.threads Developer
Joined: Apr 2004
Posts: 1,405
have you tried using the ftp cli to list ranges of files within the directory? I'm not sure what your needs are, but ftp is a cli that is more commonly accessed by end-users in this day and age with a gui, such as the for mentioned filezilla.


isaac @ id242.com // my forum @ CelicaHobby.com
a current developer of UBB.threads php forum software // 7.7.2 Progress Notes
Re: 7.6.2 file attachment cleanup [Re: Zarzal] #262023 01/09/2019 6:22 PM
Joined: Dec 2003
Posts: 5,891
Ruben Offline
Offline
Joined: Dec 2003
Posts: 5,891
Don't fret over it.
It was at lest 14 months ago or so
And the customer ended up going to another platform..
I don't even recall what he migrated to..
But he got somebody to do it..


Blue Man Group
There is no such thing as stupid questions. Just stupid answers

Additional Community Resources
UBB Developers
UBB Wiki
ShoutChat Box
Recent Topics
Forum Not Working after Cpanel upgrade.
by doug - 06/14/2019 2:35 PM
YouTube does not display?
by BlackMale - 06/09/2019 6:34 AM
Removing a redirect?
by Baldeagle - 06/01/2019 4:50 PM
Mobile Version Not so Friendly?
by PianoWorld - 05/28/2019 6:50 PM
Who's Online Now
2 registered members (isaac, UBBSystems), 75 guests, and 162 spiders.
Key: Admin, Global Mod, Mod
Member Spotlight
Posts: 22
Joined: June 2008
Show All Member Profiles 
Forum Statistics
Forums35
Topics35,302
Posts192,517
Members12,169
Most Online978
Jun 24th, 2007
Random Gallery Image
Latest Photos
Land Rocks
Cozy Places
Trees
Retreats
Powered by UBB.threads™ PHP Forum Software 7.7.2
(Snapshot build 20190617)